Jul 22 16:07:16.485884 ip-10-0-143-222 systemd[1]: Starting Kubernetes Kubelet... Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.883415 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888161 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888172 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888176 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888179 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888182 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888185 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888187 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:07:16.963908 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888191 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:07:16.952187 ip-10-0-143-222 systemd[1]: Started Kubernetes Kubelet. Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888194 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888196 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888199 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888201 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888209 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888212 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888214 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888216 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888219 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888221 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888224 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888227 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888229 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888232 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888234 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888236 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888239 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888242 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:07:16.965433 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888244 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888247 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888249 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888252 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888254 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888257 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888259 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888261 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888264 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888266 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888269 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888271 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888274 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888277 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888281 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888285 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888288 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888291 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888294 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:07:16.966611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888298 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888301 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888303 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888306 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888310 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888313 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888316 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888318 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888320 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888323 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888326 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888328 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888331 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888333 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888336 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888339 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888342 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888344 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888347 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888349 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:07:16.967617 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888352 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888354 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888357 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888359 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888362 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888364 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888368 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888371 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888374 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888377 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888381 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888384 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888387 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888390 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888392 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888395 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888397 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888400 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888402 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:07:16.968446 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888404 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888407 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888409 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888412 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888414 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888417 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888419 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888421 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888424 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888426 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888428 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888431 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888434 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888436 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.888440 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889085 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889092 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889095 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889098 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889100 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:07:16.969429 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889103 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889106 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889108 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889111 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889116 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889120 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889123 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889126 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889129 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889132 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889135 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889138 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889140 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889143 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889146 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889148 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889151 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889153 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889156 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:07:16.970408 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889158 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889160 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889163 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889165 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889168 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889170 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889173 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889176 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889178 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889180 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889183 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889185 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889188 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889190 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889193 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889195 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889197 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889200 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889203 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889205 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:07:16.971397 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889207 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889211 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889213 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889216 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889218 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889221 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889223 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889226 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889228 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889230 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889233 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889235 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889238 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889240 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889242 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889245 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889247 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889249 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889252 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889254 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:07:16.972352 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889257 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889260 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889262 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889265 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889267 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889269 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889272 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889275 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889278 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889282 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889285 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889288 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889290 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889293 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889296 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889299 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889302 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889305 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889307 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889310 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:07:16.973143 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889312 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889315 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889318 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889321 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889323 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889326 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889328 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889330 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889333 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889335 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889339 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889342 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889344 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889347 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889349 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889698 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889703 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889706 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889709 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889712 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:07:16.974158 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889715 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889718 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889721 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889724 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889727 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889730 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889732 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889735 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889737 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889740 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889742 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889745 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889747 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889750 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889752 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889754 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889757 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889759 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889762 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889764 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:07:16.975183 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889766 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889769 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889771 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889773 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889776 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889779 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889781 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889784 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889787 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889789 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889792 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889795 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889799 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889802 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889805 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889808 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889811 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889814 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889816 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889821 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:07:16.976084 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889823 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889826 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889829 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889831 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889834 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889836 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889840 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889843 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889845 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889848 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889850 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889853 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889855 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889857 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889860 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889862 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889865 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889867 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889869 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889872 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:07:16.977009 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889875 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889877 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889879 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889882 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889884 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889886 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889889 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889891 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889893 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889896 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889899 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889906 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889909 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889912 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889915 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889919 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889922 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889925 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889929 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:07:16.977580 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889932 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889935 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889937 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889941 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889944 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889946 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889948 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889951 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889953 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889955 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889958 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889961 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889963 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889966 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.889968 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890033 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890039 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890045 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890048 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890052 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890055 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 16:07:16.978160 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890058 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890062 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890065 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890070 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890073 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890077 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890079 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890082 2564 flags.go:64] FLAG: --cgroup-root="" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890085 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890087 2564 flags.go:64] FLAG: --client-ca-file="" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890093 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890096 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890099 2564 flags.go:64] FLAG: --cluster-domain="" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890102 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890105 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890108 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890111 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890114 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890117 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890120 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890123 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890126 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890129 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890132 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 16:07:16.978797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890135 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890137 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890141 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890144 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890146 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890149 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890152 2564 flags.go:64] FLAG: --enable-server="true" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890154 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890158 2564 flags.go:64] FLAG: --event-burst="100" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890161 2564 flags.go:64] FLAG: --event-qps="50" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890164 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890169 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890172 2564 flags.go:64] FLAG: --eviction-hard="" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890176 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890179 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890182 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890184 2564 flags.go:64] FLAG: --eviction-soft="" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890187 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890191 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890194 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890197 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890199 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890202 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890205 2564 flags.go:64] FLAG: --feature-gates="" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890215 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 16:07:16.979522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890218 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890221 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890224 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890227 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890230 2564 flags.go:64] FLAG: --help="false" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890232 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-143-222.ec2.internal" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890235 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890238 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890241 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890244 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890247 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890250 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890252 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890255 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890258 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890260 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890263 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890266 2564 flags.go:64] FLAG: --kube-reserved="" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890270 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890273 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890276 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890278 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890281 2564 flags.go:64] FLAG: --lock-file="" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890284 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 16:07:16.982662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890286 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890290 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890296 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890298 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890301 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890304 2564 flags.go:64] FLAG: --logging-format="text" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890306 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890310 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890313 2564 flags.go:64] FLAG: --manifest-url="" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890315 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890319 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890321 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890325 2564 flags.go:64] FLAG: --max-pods="110" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890328 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890330 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890333 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890336 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890339 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890341 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890344 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890350 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890353 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890355 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 16:07:16.983849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890358 2564 flags.go:64] FLAG: --pod-cidr="" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890361 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890366 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890370 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890373 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890376 2564 flags.go:64] FLAG: --port="10250" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890379 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890382 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0eb36d46dd5da1294" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890385 2564 flags.go:64] FLAG: --qos-reserved="" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890387 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890390 2564 flags.go:64] FLAG: --register-node="true" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890394 2564 flags.go:64] FLAG: --register-with-taints="" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890398 2564 flags.go:64] FLAG: --registry-burst="10" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890400 2564 flags.go:64] FLAG: --registry-qps="5" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890403 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890406 2564 flags.go:64] FLAG: --reserved-memory="" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890409 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890412 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890415 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890417 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890420 2564 flags.go:64] FLAG: --runonce="false" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890423 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890426 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890428 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890431 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 16:07:17.125884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890434 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890436 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890439 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890442 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890445 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890447 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890450 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890453 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890455 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890458 2564 flags.go:64] FLAG: --system-cgroups="" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890462 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890467 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890470 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890473 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890895 2564 flags.go:64] FLAG: --tls-min-version="" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890900 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890904 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890908 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890914 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890918 2564 flags.go:64] FLAG: --v="2" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890924 2564 flags.go:64] FLAG: --version="false" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890929 2564 flags.go:64] FLAG: --vmodule="" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890934 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.890938 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891090 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:07:17.143736 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891096 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891100 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891104 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891108 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891111 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891115 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891119 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891124 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891130 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891134 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891138 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891141 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891145 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891148 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891152 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891156 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891159 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891164 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891168 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891171 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:07:17.144571 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891174 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891179 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891183 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891186 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891190 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891195 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891198 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891202 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891206 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891210 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891213 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891218 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891221 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891225 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891229 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891232 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891236 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891240 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891243 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891247 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:07:17.145378 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891251 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891254 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891260 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891265 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891269 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891273 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891276 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891280 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891284 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891288 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891292 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891295 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891299 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891303 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891307 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891311 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891315 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891320 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891324 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:07:17.145976 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891327 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891331 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891334 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891338 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891341 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891345 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891350 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891354 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891357 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891361 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891365 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891368 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891372 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891376 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891379 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891382 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891386 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891390 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891393 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:07:17.146587 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891396 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891400 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891403 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891408 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891412 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891415 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891421 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891425 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891429 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891434 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891438 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891443 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891447 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891451 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891455 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891458 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891462 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891466 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891469 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:07:17.147218 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.891473 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.892040 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.898741 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.898755 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.898772 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.898780 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898858 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898863 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898867 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898870 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898873 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898876 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898879 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898882 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898884 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:07:17.147828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898887 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898889 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898892 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898898 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898901 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898903 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898905 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898914 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898916 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898919 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898923 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898926 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898930 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898932 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898935 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898937 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898940 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898942 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898945 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:07:17.148291 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898948 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898950 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898953 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898955 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898958 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898960 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898963 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898965 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898968 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898970 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898972 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898975 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898977 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898980 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898982 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898984 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898988 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898991 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898993 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:07:17.148841 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.898996 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899002 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899010 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899013 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899015 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899018 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899020 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899023 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899025 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899027 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899030 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899032 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899035 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899038 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899041 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899044 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899047 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899050 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899053 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:07:17.149344 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899058 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899061 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899063 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899066 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899068 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899071 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899073 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899075 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899078 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899080 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899084 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899086 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899089 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899091 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899094 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899096 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899104 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899107 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899109 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899112 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:07:17.149937 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899114 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899117 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899119 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899122 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899124 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899126 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899129 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899132 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899136 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899139 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899141 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899143 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899147 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.899152 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899285 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:07:17.150455 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899289 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899292 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899294 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899296 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899299 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899301 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899304 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899307 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899310 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899313 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899315 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899318 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899320 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899322 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899332 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899335 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899337 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899340 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899342 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899344 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:07:17.150930 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899347 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899349 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899352 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899354 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899356 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899360 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899364 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899366 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899369 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899373 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899376 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899378 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899381 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899384 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899386 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899389 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899391 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899394 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899396 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:07:17.151486 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899398 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899402 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899405 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899407 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899410 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899412 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899414 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899417 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899425 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899428 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899430 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899433 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899435 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899438 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899440 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899443 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899445 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899447 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899450 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899452 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:07:17.152046 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899455 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899457 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899461 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899464 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899466 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899468 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899470 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899473 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899475 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899477 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899480 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899482 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899485 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899488 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899491 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899493 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899496 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899498 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899500 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899503 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:07:17.152559 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899505 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899513 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899516 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899518 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899521 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899523 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899526 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899528 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899532 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899535 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899538 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899540 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899543 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899545 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899549 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899551 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899553 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899556 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:07:17.153132 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:16.899560 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.899564 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.899682 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.901646 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.902949 2564 server.go:1019] "Starting client certificate rotation" Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.903043 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.903783 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.905393 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.908390 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.916055 2564 log.go:25] "Validated CRI v1 runtime API" Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.920278 2564 log.go:25] "Validated CRI v1 image API" Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.921581 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 16:07:17.153591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.924025 2564 fs.go:135] Filesystem UUIDs: map[0b165e00-831f-4088-babb-78c4b520d1a6:/dev/nvme0n1p3 37ea14b7-26ad-493a-bf42-2483fe28c559:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 22 16:07:17.154030 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.924043 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 16:07:17.154402 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.930037 2564 manager.go:216] Machine: {Timestamp:2026-07-22 16:07:16.928259748 +0000 UTC m=+0.323801574 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3217951 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2692d2f6e0ccded0eae475abb12a21 SystemUUID:ec2692d2-f6e0-ccde-d0ea-e475abb12a21 BootID:c8e2ad09-dc6f-4624-8dbd-b4f111e77f00 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2d:68:d8:3a:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2d:68:d8:3a:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:bd:25:0b:97:71 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 16:07:17.154402 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.930129 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 16:07:17.154402 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.930213 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 16:07:17.154402 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.933705 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 16:07:17.154402 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.933725 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-222.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 16:07:17.154402 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.934393 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 16:07:17.154402 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.934402 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 16:07:17.154402 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.934409 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.935096 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.936280 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.939180 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.939307 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.939989 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.940007 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.940024 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.940035 2564 kubelet.go:398] "Adding apiserver pod source" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.940045 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.940353 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.940369 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.944046 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.945782 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.945807 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.946780 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.946792 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.946798 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.946803 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.946812 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.946817 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.946920 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.946925 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.947127 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.947135 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.947348 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.947354 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.948641 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.948657 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.951470 2564 server.go:1268] "Started kubelet" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.951637 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.951646 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.951714 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.953222 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.954384 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.954730 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:16.957340 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:16.958919 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-222.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:16.958949 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 16:07:17.154750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.958979 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-222.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:16.958920 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-222.ec2.internal.18c4a8555e2febe2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-222.ec2.internal,UID:ip-10-0-143-222.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-222.ec2.internal,},FirstTimestamp:2026-07-22 16:07:16.95144445 +0000 UTC m=+0.346986273,LastTimestamp:2026-07-22 16:07:16.95144445 +0000 UTC m=+0.346986273,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-222.ec2.internal,}" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.960576 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.960583 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:16.960670 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.961214 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.961215 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.961247 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.961313 2564 factory.go:55] Registering systemd factory Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.961330 2564 factory.go:223] Registration of the systemd container factory successfully Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.961361 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.961368 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.962178 2564 factory.go:153] Registering CRI-O factory Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.962189 2564 factory.go:223] Registration of the crio container factory successfully Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.962227 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.962243 2564 factory.go:103] Registering Raw factory Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.962254 2564 manager.go:1196] Started watching for new ooms in manager Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.962607 2564 manager.go:349] Starting recovery of all containers Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:16.962760 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-222.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:16.962974 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.976397 2564 manager.go:354] Recovery completed Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.980098 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.986171 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.986195 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.986205 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.986618 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.986644 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.986647 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.986659 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.987815 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.987839 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.987850 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.987878 2564 policy_none.go:49] "None policy: Start" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.987890 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 16:07:17.316943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.987901 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:16.989433 2564 policy_none.go:47] "Start" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.003890 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-222.ec2.internal.18c4a8556041fc26 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-222.ec2.internal,UID:ip-10-0-143-222.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-222.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-222.ec2.internal,},FirstTimestamp:2026-07-22 16:07:16.986182694 +0000 UTC m=+0.381724518,LastTimestamp:2026-07-22 16:07:16.986182694 +0000 UTC m=+0.381724518,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-222.ec2.internal,}" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.007118 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6mqcn" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.008489 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-222.ec2.internal.18c4a85560423ee2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-222.ec2.internal,UID:ip-10-0-143-222.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-222.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-222.ec2.internal,},FirstTimestamp:2026-07-22 16:07:16.986199778 +0000 UTC m=+0.381741601,LastTimestamp:2026-07-22 16:07:16.986199778 +0000 UTC m=+0.381741601,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-222.ec2.internal,}" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.014246 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-222.ec2.internal.18c4a855604261ea default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-222.ec2.internal,UID:ip-10-0-143-222.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-222.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-222.ec2.internal,},FirstTimestamp:2026-07-22 16:07:16.986208746 +0000 UTC m=+0.381750569,LastTimestamp:2026-07-22 16:07:16.986208746 +0000 UTC m=+0.381750569,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-222.ec2.internal,}" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.016926 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6mqcn" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.022098 2564 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-143-222.ec2.internal.18c4a8556041fc26\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-222.ec2.internal.18c4a8556041fc26 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-222.ec2.internal,UID:ip-10-0-143-222.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-222.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-222.ec2.internal,},FirstTimestamp:2026-07-22 16:07:16.986182694 +0000 UTC m=+0.381724518,LastTimestamp:2026-07-22 16:07:16.987826773 +0000 UTC m=+0.383368596,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-222.ec2.internal,}" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.025128 2564 manager.go:337] "Starting Device Plugin manager" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.025152 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.025160 2564 server.go:87] "Starting device plugin registration server" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.025324 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 16:07:17.318359 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.025335 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.025394 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.025464 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.025472 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.025919 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.026086 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.120668 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.121781 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.121810 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.121830 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.121863 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.124006 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.125537 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.126219 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.126241 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.126251 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.126270 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.141645 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.141660 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-222.ec2.internal\": node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.164543 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.222585 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal"] Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.222670 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.223431 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.223458 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.223472 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.224744 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.224851 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.224874 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.225358 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.225384 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.225395 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.225359 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.225457 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.225471 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.226868 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.318834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.226895 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:07:17.319865 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.227523 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:07:17.319865 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.227553 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:07:17.319865 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.227565 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:07:17.319865 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.263056 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94a4e9989bd9a00f5f1b360ffb81069d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal\" (UID: \"94a4e9989bd9a00f5f1b360ffb81069d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.319865 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.263079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94a4e9989bd9a00f5f1b360ffb81069d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal\" (UID: \"94a4e9989bd9a00f5f1b360ffb81069d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.319865 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.263096 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a761dd84e540e2dc089b8031677b8ec6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-222.ec2.internal\" (UID: \"a761dd84e540e2dc089b8031677b8ec6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.319865 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.263110 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-222.ec2.internal\" not found" node="ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.319865 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.264929 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:17.319865 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.267059 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-222.ec2.internal\" not found" node="ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.495135 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.363756 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94a4e9989bd9a00f5f1b360ffb81069d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal\" (UID: \"94a4e9989bd9a00f5f1b360ffb81069d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.495135 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.363789 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94a4e9989bd9a00f5f1b360ffb81069d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal\" (UID: \"94a4e9989bd9a00f5f1b360ffb81069d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.495135 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.363812 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a761dd84e540e2dc089b8031677b8ec6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-222.ec2.internal\" (UID: \"a761dd84e540e2dc089b8031677b8ec6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.495135 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.363860 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a761dd84e540e2dc089b8031677b8ec6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-222.ec2.internal\" (UID: \"a761dd84e540e2dc089b8031677b8ec6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.495135 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.363866 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94a4e9989bd9a00f5f1b360ffb81069d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal\" (UID: \"94a4e9989bd9a00f5f1b360ffb81069d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.495135 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.363866 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/94a4e9989bd9a00f5f1b360ffb81069d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal\" (UID: \"94a4e9989bd9a00f5f1b360ffb81069d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.495135 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.365907 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:17.495135 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.466035 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:17.666999 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.566710 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:17.666999 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.566763 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.666999 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.570385 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal" Jul 22 16:07:17.666999 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.666905 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:17.838015 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.767123 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:18.016139 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.868165 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:18.016139 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.902559 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 16:07:18.016139 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:17.902709 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 16:07:18.016139 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:17.968562 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:18.021038 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.020565 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 16:02:17 +0000 UTC" deadline="2028-02-14 06:33:48.802312194 +0000 UTC" Jul 22 16:07:18.021038 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.020603 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13718h26m30.781713013s" Jul 22 16:07:18.207947 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:18.068763 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:18.207947 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.111937 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:07:18.207947 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.127237 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:07:18.207947 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.128178 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:07:18.207947 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.128212 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:07:18.207947 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.128226 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:07:18.207947 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:18.169231 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-222.ec2.internal\" not found" Jul 22 16:07:18.207947 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:18.199176 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a4e9989bd9a00f5f1b360ffb81069d.slice/crio-15261d1848fe84d54faef544707df39f118bcd5c2227d7c519840c981a78e073 WatchSource:0}: Error finding container 15261d1848fe84d54faef544707df39f118bcd5c2227d7c519840c981a78e073: Status 404 returned error can't find the container with id 15261d1848fe84d54faef544707df39f118bcd5c2227d7c519840c981a78e073 Jul 22 16:07:18.207947 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:18.199745 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda761dd84e540e2dc089b8031677b8ec6.slice/crio-1a5ba2c138a67d2b287fa819ed6defff0f0a03c036b374ab39c118baf650de0b WatchSource:0}: Error finding container 1a5ba2c138a67d2b287fa819ed6defff0f0a03c036b374ab39c118baf650de0b: Status 404 returned error can't find the container with id 1a5ba2c138a67d2b287fa819ed6defff0f0a03c036b374ab39c118baf650de0b Jul 22 16:07:18.207947 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.203849 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:07:18.417282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.237538 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:07:18.417282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.262082 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" Jul 22 16:07:18.417282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.286238 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 16:07:18.417282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.288187 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal" Jul 22 16:07:18.417282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.301189 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 16:07:18.417282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.409743 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.943730 2564 apiserver.go:52] "Watching apiserver" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.958218 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.959692 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7zwc4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798","openshift-dns/node-resolver-hb82z","openshift-image-registry/node-ca-nrhcg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal","openshift-multus/multus-4tmkn","openshift-multus/network-metrics-daemon-lfc4g","openshift-network-diagnostics/network-check-target-5sllq","kube-system/konnectivity-agent-vvbs2","kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal","openshift-cluster-node-tuning-operator/tuned-sdbnq","openshift-multus/multus-additional-cni-plugins-p8jnb","openshift-network-operator/iptables-alerter-rhlwc","openshift-ovn-kubernetes/ovnkube-node-lsfmv"] Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.962205 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.962391 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:18.962480 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.965258 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.967765 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.970112 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.970771 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.970816 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.971512 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-99sks\"" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.972727 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.973356 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.973385 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp6ml\" (UniqueName: \"kubernetes.io/projected/02186816-0aad-4a17-943b-e09d6dca7a03-kube-api-access-bp6ml\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.973762 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.973768 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.973826 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6m6lz\"" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.974139 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.974481 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.974504 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.975303 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4tmkn" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.975344 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-g8hlf\"" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.977926 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:18.988102 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.978424 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.978506 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.978618 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.978861 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.978874 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.981420 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.982009 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.983249 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q49f7\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.983307 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.983398 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.983439 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.983505 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.983741 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.983757 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v5rm7\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.983830 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.985390 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.986620 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mc5d4\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.987960 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:18.988012 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.988703 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 16:07:18.990058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.988703 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 16:07:18.992659 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.992021 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:18.992659 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:18.992112 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:18.992659 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.992381 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qsrjk\"" Jul 22 16:07:18.992659 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.992551 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 16:07:18.993337 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.992739 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 16:07:18.993337 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.993006 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:07:18.993337 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.993259 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n9fbq\"" Jul 22 16:07:18.995158 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.993978 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 16:07:18.996521 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.996403 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:18.998305 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.998185 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 16:07:18.999400 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:18.999381 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 16:07:19.000590 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.000565 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 16:07:19.000868 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.000852 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sfcqh\"" Jul 22 16:07:19.036705 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.036429 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7g99w" Jul 22 16:07:19.048601 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.048577 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7g99w" Jul 22 16:07:19.062862 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.062800 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 16:07:19.073576 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073556 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-modprobe-d\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.073787 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073600 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-cni-dir\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.073787 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073644 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:19.073787 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073671 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-tuned\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.073787 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8thb\" (UniqueName: \"kubernetes.io/projected/e98f2ec9-31ce-45fd-a11f-9bb46626c62b-kube-api-access-z8thb\") pod \"node-resolver-hb82z\" (UID: \"e98f2ec9-31ce-45fd-a11f-9bb46626c62b\") " pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:19.073787 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073723 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-kubelet\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.073787 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073747 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-cni-bin\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073787 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b9601bb1-b7f1-4030-82ee-412a348ea931-env-overrides\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073824 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b9601bb1-b7f1-4030-82ee-412a348ea931-ovnkube-script-lib\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073852 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpvf6\" (UniqueName: \"kubernetes.io/projected/b9601bb1-b7f1-4030-82ee-412a348ea931-kube-api-access-wpvf6\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073879 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-cnibin\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073904 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91b8bd81-42f0-4480-9f6f-cdac9397a125-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073938 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b9601bb1-b7f1-4030-82ee-412a348ea931-ovnkube-config\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073968 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-system-cni-dir\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.073992 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/313873ed-89d3-43ad-b540-633cb5cbe79a-cni-binary-copy\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074017 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-var-lib-cni-bin\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074042 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-var-lib-kubelet\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.074095 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074074 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nq26\" (UniqueName: \"kubernetes.io/projected/a7787dd1-9dde-4848-a86b-1076da81f7a0-kube-api-access-7nq26\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074114 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-etc-openvswitch\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074144 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-run-ovn\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074177 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-run\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074200 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d64c7c2f-b2b4-4738-88dd-e56aeeadd228-serviceca\") pod \"node-ca-nrhcg\" (UID: \"d64c7c2f-b2b4-4738-88dd-e56aeeadd228\") " pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074237 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-run-netns\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074273 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-node-log\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074300 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-run-k8s-cni-cncf-io\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074324 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5v4q\" (UniqueName: \"kubernetes.io/projected/d64c7c2f-b2b4-4738-88dd-e56aeeadd228-kube-api-access-h5v4q\") pod \"node-ca-nrhcg\" (UID: \"d64c7c2f-b2b4-4738-88dd-e56aeeadd228\") " pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074350 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-cni-netd\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074376 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-registration-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074414 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-run-systemd\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074443 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074478 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-hostroot\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074516 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-etc-selinux\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074601 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g4rj\" (UniqueName: \"kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj\") pod \"network-check-target-5sllq\" (UID: \"3e191455-0b73-49ef-821c-1c6e74d88494\") " pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-os-release\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074672 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95tl8\" (UniqueName: \"kubernetes.io/projected/91b8bd81-42f0-4480-9f6f-cdac9397a125-kube-api-access-95tl8\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074694 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-var-lib-openvswitch\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074709 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-etc-kubernetes\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074727 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d67b9e6-5f4b-421a-af5f-7d8555f9f726-iptables-alerter-script\") pod \"iptables-alerter-rhlwc\" (UID: \"2d67b9e6-5f4b-421a-af5f-7d8555f9f726\") " pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074743 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-dbus\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074764 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-systemd\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074786 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d64c7c2f-b2b4-4738-88dd-e56aeeadd228-host\") pod \"node-ca-nrhcg\" (UID: \"d64c7c2f-b2b4-4738-88dd-e56aeeadd228\") " pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074813 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-run-ovn-kubernetes\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074830 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-conf-dir\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.074996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074859 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-device-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074877 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e89a06fb-1480-471d-be5d-9758ece1a6f8-agent-certs\") pod \"konnectivity-agent-vvbs2\" (UID: \"e89a06fb-1480-471d-be5d-9758ece1a6f8\") " pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074922 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp6ml\" (UniqueName: \"kubernetes.io/projected/02186816-0aad-4a17-943b-e09d6dca7a03-kube-api-access-bp6ml\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-lib-modules\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074966 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7787dd1-9dde-4848-a86b-1076da81f7a0-tmp\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074981 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-daemon-config\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.074997 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-sys\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075052 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-systemd-units\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075090 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-run-netns\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-run-multus-certs\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.075138 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-cnibin\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075169 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-kubernetes\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.075728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075194 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-sysctl-d\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.075221 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs podName:02186816-0aad-4a17-943b-e09d6dca7a03 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:19.57519865 +0000 UTC m=+2.970740476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs") pod "network-metrics-daemon-lfc4g" (UID: "02186816-0aad-4a17-943b-e09d6dca7a03") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075263 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-sysctl-conf\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e98f2ec9-31ce-45fd-a11f-9bb46626c62b-tmp-dir\") pod \"node-resolver-hb82z\" (UID: \"e98f2ec9-31ce-45fd-a11f-9bb46626c62b\") " pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075314 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-run-openvswitch\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075336 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-sys-fs\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075359 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d67b9e6-5f4b-421a-af5f-7d8555f9f726-host-slash\") pod \"iptables-alerter-rhlwc\" (UID: \"2d67b9e6-5f4b-421a-af5f-7d8555f9f726\") " pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.075384 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-slash\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076110 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9601bb1-b7f1-4030-82ee-412a348ea931-ovn-node-metrics-cert\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076152 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-var-lib-cni-multus\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076176 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076200 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-socket-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076225 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-kubelet-config\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:19.076463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076249 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076274 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-var-lib-kubelet\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076300 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e98f2ec9-31ce-45fd-a11f-9bb46626c62b-hosts-file\") pod \"node-resolver-hb82z\" (UID: \"e98f2ec9-31ce-45fd-a11f-9bb46626c62b\") " pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-log-socket\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076347 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e89a06fb-1480-471d-be5d-9758ece1a6f8-konnectivity-ca\") pod \"konnectivity-agent-vvbs2\" (UID: \"e89a06fb-1480-471d-be5d-9758ece1a6f8\") " pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076371 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jqv7\" (UniqueName: \"kubernetes.io/projected/2d67b9e6-5f4b-421a-af5f-7d8555f9f726-kube-api-access-8jqv7\") pod \"iptables-alerter-rhlwc\" (UID: \"2d67b9e6-5f4b-421a-af5f-7d8555f9f726\") " pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076399 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-sysconfig\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-host\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076445 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-socket-dir-parent\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076485 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp829\" (UniqueName: \"kubernetes.io/projected/313873ed-89d3-43ad-b540-633cb5cbe79a-kube-api-access-lp829\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076514 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sjlh\" (UniqueName: \"kubernetes.io/projected/52990c4f-55be-4b7b-a825-a73f7caf7f08-kube-api-access-6sjlh\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076538 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-system-cni-dir\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076563 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91b8bd81-42f0-4480-9f6f-cdac9397a125-cni-binary-copy\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.077054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076589 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91b8bd81-42f0-4480-9f6f-cdac9397a125-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.077765 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.076617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-os-release\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.090993 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.090960 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 16:07:19.095182 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.095161 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp6ml\" (UniqueName: \"kubernetes.io/projected/02186816-0aad-4a17-943b-e09d6dca7a03-kube-api-access-bp6ml\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:19.127350 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.127302 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" event={"ID":"94a4e9989bd9a00f5f1b360ffb81069d","Type":"ContainerStarted","Data":"15261d1848fe84d54faef544707df39f118bcd5c2227d7c519840c981a78e073"} Jul 22 16:07:19.128779 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.128747 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal" event={"ID":"a761dd84e540e2dc089b8031677b8ec6","Type":"ContainerStarted","Data":"1a5ba2c138a67d2b287fa819ed6defff0f0a03c036b374ab39c118baf650de0b"} Jul 22 16:07:19.176862 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.176838 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-sysconfig\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.177034 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.176872 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-host\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.177034 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.176899 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-socket-dir-parent\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.177034 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.176924 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp829\" (UniqueName: \"kubernetes.io/projected/313873ed-89d3-43ad-b540-633cb5cbe79a-kube-api-access-lp829\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.177034 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.176941 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-sysconfig\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.177034 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.176974 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sjlh\" (UniqueName: \"kubernetes.io/projected/52990c4f-55be-4b7b-a825-a73f7caf7f08-kube-api-access-6sjlh\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.177034 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.176995 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-host\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.177034 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177014 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-system-cni-dir\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177039 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91b8bd81-42f0-4480-9f6f-cdac9397a125-cni-binary-copy\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177059 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91b8bd81-42f0-4480-9f6f-cdac9397a125-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177055 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-socket-dir-parent\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-os-release\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177127 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-system-cni-dir\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177156 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-os-release\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177173 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-modprobe-d\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177203 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-cni-dir\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177303 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-modprobe-d\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177313 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-tuned\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.177335 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:19.177379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177350 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-cni-dir\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.178039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177408 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8thb\" (UniqueName: \"kubernetes.io/projected/e98f2ec9-31ce-45fd-a11f-9bb46626c62b-kube-api-access-z8thb\") pod \"node-resolver-hb82z\" (UID: \"e98f2ec9-31ce-45fd-a11f-9bb46626c62b\") " pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:19.178039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177435 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-kubelet\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177842 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91b8bd81-42f0-4480-9f6f-cdac9397a125-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.178039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177917 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-cni-bin\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b9601bb1-b7f1-4030-82ee-412a348ea931-env-overrides\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178039 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.177962 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret podName:7ce617c4-ea42-4943-8dde-54b9bfc24d0d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:19.677940801 +0000 UTC m=+3.073482617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret") pod "global-pull-secret-syncer-7zwc4" (UID: "7ce617c4-ea42-4943-8dde-54b9bfc24d0d") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:19.178039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.177999 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b9601bb1-b7f1-4030-82ee-412a348ea931-ovnkube-script-lib\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178009 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-cni-bin\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178437 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178030 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpvf6\" (UniqueName: \"kubernetes.io/projected/b9601bb1-b7f1-4030-82ee-412a348ea931-kube-api-access-wpvf6\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178437 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-cnibin\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.178437 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178301 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-kubelet\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178437 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178308 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91b8bd81-42f0-4480-9f6f-cdac9397a125-cni-binary-copy\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.178437 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178343 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91b8bd81-42f0-4480-9f6f-cdac9397a125-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.178437 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178378 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b9601bb1-b7f1-4030-82ee-412a348ea931-ovnkube-config\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178437 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178384 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-cnibin\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178468 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-system-cni-dir\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/313873ed-89d3-43ad-b540-633cb5cbe79a-cni-binary-copy\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178508 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b9601bb1-b7f1-4030-82ee-412a348ea931-env-overrides\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178528 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-var-lib-cni-bin\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178556 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-var-lib-kubelet\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178581 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-system-cni-dir\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178600 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nq26\" (UniqueName: \"kubernetes.io/projected/a7787dd1-9dde-4848-a86b-1076da81f7a0-kube-api-access-7nq26\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-etc-openvswitch\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178653 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-run-ovn\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178663 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-var-lib-kubelet\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178676 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-run\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178717 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d64c7c2f-b2b4-4738-88dd-e56aeeadd228-serviceca\") pod \"node-ca-nrhcg\" (UID: \"d64c7c2f-b2b4-4738-88dd-e56aeeadd228\") " pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178744 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-etc-openvswitch\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-run-netns\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.178839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-run-netns\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178801 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-node-log\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178628 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-var-lib-cni-bin\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178828 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-run-k8s-cni-cncf-io\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178854 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5v4q\" (UniqueName: \"kubernetes.io/projected/d64c7c2f-b2b4-4738-88dd-e56aeeadd228-kube-api-access-h5v4q\") pod \"node-ca-nrhcg\" (UID: \"d64c7c2f-b2b4-4738-88dd-e56aeeadd228\") " pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178880 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-cni-netd\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178902 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-run\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178906 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-registration-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178906 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b9601bb1-b7f1-4030-82ee-412a348ea931-ovnkube-config\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178925 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91b8bd81-42f0-4480-9f6f-cdac9397a125-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178945 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-run-systemd\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178959 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-node-log\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178973 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179002 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-hostroot\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.179667 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-etc-selinux\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179001 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-run-k8s-cni-cncf-io\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179059 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4rj\" (UniqueName: \"kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj\") pod \"network-check-target-5sllq\" (UID: \"3e191455-0b73-49ef-821c-1c6e74d88494\") " pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179084 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-os-release\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179091 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/313873ed-89d3-43ad-b540-633cb5cbe79a-cni-binary-copy\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179112 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95tl8\" (UniqueName: \"kubernetes.io/projected/91b8bd81-42f0-4480-9f6f-cdac9397a125-kube-api-access-95tl8\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179138 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-var-lib-openvswitch\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179161 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-hostroot\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179162 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-etc-kubernetes\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179202 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-etc-kubernetes\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d67b9e6-5f4b-421a-af5f-7d8555f9f726-iptables-alerter-script\") pod \"iptables-alerter-rhlwc\" (UID: \"2d67b9e6-5f4b-421a-af5f-7d8555f9f726\") " pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.178857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-run-ovn\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-dbus\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-cni-netd\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179283 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-systemd\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.180356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179333 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d64c7c2f-b2b4-4738-88dd-e56aeeadd228-serviceca\") pod \"node-ca-nrhcg\" (UID: \"d64c7c2f-b2b4-4738-88dd-e56aeeadd228\") " pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179342 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d64c7c2f-b2b4-4738-88dd-e56aeeadd228-host\") pod \"node-ca-nrhcg\" (UID: \"d64c7c2f-b2b4-4738-88dd-e56aeeadd228\") " pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179401 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d64c7c2f-b2b4-4738-88dd-e56aeeadd228-host\") pod \"node-ca-nrhcg\" (UID: \"d64c7c2f-b2b4-4738-88dd-e56aeeadd228\") " pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179402 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-registration-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179414 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-os-release\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179424 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-run-ovn-kubernetes\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179443 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b9601bb1-b7f1-4030-82ee-412a348ea931-ovnkube-script-lib\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179455 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-conf-dir\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179481 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-device-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179506 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e89a06fb-1480-471d-be5d-9758ece1a6f8-agent-certs\") pod \"konnectivity-agent-vvbs2\" (UID: \"e89a06fb-1480-471d-be5d-9758ece1a6f8\") " pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179510 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-run-ovn-kubernetes\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179533 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-lib-modules\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179559 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7787dd1-9dde-4848-a86b-1076da81f7a0-tmp\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179585 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-run-systemd\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179589 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-daemon-config\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.181107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179595 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-var-lib-openvswitch\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179642 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-sys\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179665 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-systemd-units\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-dbus\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179692 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-run-netns\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179678 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d67b9e6-5f4b-421a-af5f-7d8555f9f726-iptables-alerter-script\") pod \"iptables-alerter-rhlwc\" (UID: \"2d67b9e6-5f4b-421a-af5f-7d8555f9f726\") " pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-run-multus-certs\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180133 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-cnibin\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-kubernetes\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180215 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-sysctl-d\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.179724 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-systemd\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180339 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-sysctl-d\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180403 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180481 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-cnibin\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180542 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-kubernetes\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.185633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180583 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-daemon-config\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180642 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-device-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180728 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-etc-selinux\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180802 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-lib-modules\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.180839 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-multus-conf-dir\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181201 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-systemd-units\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181258 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-sys\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181289 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-run-netns\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181332 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-run-multus-certs\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181414 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-sysctl-conf\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181458 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e98f2ec9-31ce-45fd-a11f-9bb46626c62b-tmp-dir\") pod \"node-resolver-hb82z\" (UID: \"e98f2ec9-31ce-45fd-a11f-9bb46626c62b\") " pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181488 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-run-openvswitch\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181521 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-sys-fs\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181557 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d67b9e6-5f4b-421a-af5f-7d8555f9f726-host-slash\") pod \"iptables-alerter-rhlwc\" (UID: \"2d67b9e6-5f4b-421a-af5f-7d8555f9f726\") " pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181601 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-slash\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.186483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181639 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9601bb1-b7f1-4030-82ee-412a348ea931-ovn-node-metrics-cert\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181667 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-var-lib-cni-multus\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181716 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181749 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-socket-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181781 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-kubelet-config\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181808 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181841 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-var-lib-kubelet\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e98f2ec9-31ce-45fd-a11f-9bb46626c62b-hosts-file\") pod \"node-resolver-hb82z\" (UID: \"e98f2ec9-31ce-45fd-a11f-9bb46626c62b\") " pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181902 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-log-socket\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e89a06fb-1480-471d-be5d-9758ece1a6f8-konnectivity-ca\") pod \"konnectivity-agent-vvbs2\" (UID: \"e89a06fb-1480-471d-be5d-9758ece1a6f8\") " pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.181965 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jqv7\" (UniqueName: \"kubernetes.io/projected/2d67b9e6-5f4b-421a-af5f-7d8555f9f726-kube-api-access-8jqv7\") pod \"iptables-alerter-rhlwc\" (UID: \"2d67b9e6-5f4b-421a-af5f-7d8555f9f726\") " pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182141 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182266 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-sysctl-conf\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182344 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7787dd1-9dde-4848-a86b-1076da81f7a0-var-lib-kubelet\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.187266 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182373 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e98f2ec9-31ce-45fd-a11f-9bb46626c62b-hosts-file\") pod \"node-resolver-hb82z\" (UID: \"e98f2ec9-31ce-45fd-a11f-9bb46626c62b\") " pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182436 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-log-socket\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182446 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e98f2ec9-31ce-45fd-a11f-9bb46626c62b-tmp-dir\") pod \"node-resolver-hb82z\" (UID: \"e98f2ec9-31ce-45fd-a11f-9bb46626c62b\") " pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182451 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-socket-dir\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182469 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-kubelet-config\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182524 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d67b9e6-5f4b-421a-af5f-7d8555f9f726-host-slash\") pod \"iptables-alerter-rhlwc\" (UID: \"2d67b9e6-5f4b-421a-af5f-7d8555f9f726\") " pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182577 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-host-slash\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182599 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9601bb1-b7f1-4030-82ee-412a348ea931-run-openvswitch\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182634 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91b8bd81-42f0-4480-9f6f-cdac9397a125-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182671 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/52990c4f-55be-4b7b-a825-a73f7caf7f08-sys-fs\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182736 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/313873ed-89d3-43ad-b540-633cb5cbe79a-host-var-lib-cni-multus\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.182977 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e89a06fb-1480-471d-be5d-9758ece1a6f8-konnectivity-ca\") pod \"konnectivity-agent-vvbs2\" (UID: \"e89a06fb-1480-471d-be5d-9758ece1a6f8\") " pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.183737 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7787dd1-9dde-4848-a86b-1076da81f7a0-tmp\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.185645 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9601bb1-b7f1-4030-82ee-412a348ea931-ovn-node-metrics-cert\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.185679 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e89a06fb-1480-471d-be5d-9758ece1a6f8-agent-certs\") pod \"konnectivity-agent-vvbs2\" (UID: \"e89a06fb-1480-471d-be5d-9758ece1a6f8\") " pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:19.188039 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.186052 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7787dd1-9dde-4848-a86b-1076da81f7a0-etc-tuned\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.188636 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.187782 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sjlh\" (UniqueName: \"kubernetes.io/projected/52990c4f-55be-4b7b-a825-a73f7caf7f08-kube-api-access-6sjlh\") pod \"aws-ebs-csi-driver-node-9z798\" (UID: \"52990c4f-55be-4b7b-a825-a73f7caf7f08\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.190381 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.190359 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8thb\" (UniqueName: \"kubernetes.io/projected/e98f2ec9-31ce-45fd-a11f-9bb46626c62b-kube-api-access-z8thb\") pod \"node-resolver-hb82z\" (UID: \"e98f2ec9-31ce-45fd-a11f-9bb46626c62b\") " pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:19.191319 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.191297 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5v4q\" (UniqueName: \"kubernetes.io/projected/d64c7c2f-b2b4-4738-88dd-e56aeeadd228-kube-api-access-h5v4q\") pod \"node-ca-nrhcg\" (UID: \"d64c7c2f-b2b4-4738-88dd-e56aeeadd228\") " pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:19.194508 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.194474 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:07:19.194508 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.194506 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:07:19.194661 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.194522 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6g4rj for pod openshift-network-diagnostics/network-check-target-5sllq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:19.194661 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.194581 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj podName:3e191455-0b73-49ef-821c-1c6e74d88494 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:19.694562905 +0000 UTC m=+3.090104737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6g4rj" (UniqueName: "kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj") pod "network-check-target-5sllq" (UID: "3e191455-0b73-49ef-821c-1c6e74d88494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:19.194831 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.194803 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpvf6\" (UniqueName: \"kubernetes.io/projected/b9601bb1-b7f1-4030-82ee-412a348ea931-kube-api-access-wpvf6\") pod \"ovnkube-node-lsfmv\" (UID: \"b9601bb1-b7f1-4030-82ee-412a348ea931\") " pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.196185 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.196150 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp829\" (UniqueName: \"kubernetes.io/projected/313873ed-89d3-43ad-b540-633cb5cbe79a-kube-api-access-lp829\") pod \"multus-4tmkn\" (UID: \"313873ed-89d3-43ad-b540-633cb5cbe79a\") " pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.196283 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.196250 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95tl8\" (UniqueName: \"kubernetes.io/projected/91b8bd81-42f0-4480-9f6f-cdac9397a125-kube-api-access-95tl8\") pod \"multus-additional-cni-plugins-p8jnb\" (UID: \"91b8bd81-42f0-4480-9f6f-cdac9397a125\") " pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.200756 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.200734 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jqv7\" (UniqueName: \"kubernetes.io/projected/2d67b9e6-5f4b-421a-af5f-7d8555f9f726-kube-api-access-8jqv7\") pod \"iptables-alerter-rhlwc\" (UID: \"2d67b9e6-5f4b-421a-af5f-7d8555f9f726\") " pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:19.200893 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.200873 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nq26\" (UniqueName: \"kubernetes.io/projected/a7787dd1-9dde-4848-a86b-1076da81f7a0-kube-api-access-7nq26\") pod \"tuned-sdbnq\" (UID: \"a7787dd1-9dde-4848-a86b-1076da81f7a0\") " pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.265853 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.265786 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:07:19.287829 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.287807 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" Jul 22 16:07:19.300969 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.300947 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hb82z" Jul 22 16:07:19.309463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.309444 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nrhcg" Jul 22 16:07:19.314987 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.314966 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:19.324808 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.324789 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4tmkn" Jul 22 16:07:19.331565 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.331546 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:19.342323 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.342306 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rhlwc" Jul 22 16:07:19.351867 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.351847 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" Jul 22 16:07:19.358836 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.358815 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" Jul 22 16:07:19.585092 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.585002 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:19.585277 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.585169 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:19.585277 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.585238 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs podName:02186816-0aad-4a17-943b-e09d6dca7a03 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:20.585218085 +0000 UTC m=+3.980759918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs") pod "network-metrics-daemon-lfc4g" (UID: "02186816-0aad-4a17-943b-e09d6dca7a03") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:19.686268 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.686230 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:19.686429 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.686373 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:19.686505 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.686432 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret podName:7ce617c4-ea42-4943-8dde-54b9bfc24d0d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:20.686418823 +0000 UTC m=+4.081960636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret") pod "global-pull-secret-syncer-7zwc4" (UID: "7ce617c4-ea42-4943-8dde-54b9bfc24d0d") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:19.786954 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:19.786911 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4rj\" (UniqueName: \"kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj\") pod \"network-check-target-5sllq\" (UID: \"3e191455-0b73-49ef-821c-1c6e74d88494\") " pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:19.787139 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.787079 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:07:19.787139 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.787097 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:07:19.787139 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.787108 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6g4rj for pod openshift-network-diagnostics/network-check-target-5sllq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:19.787283 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:19.787165 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj podName:3e191455-0b73-49ef-821c-1c6e74d88494 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:20.787146353 +0000 UTC m=+4.182688173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g4rj" (UniqueName: "kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj") pod "network-check-target-5sllq" (UID: "3e191455-0b73-49ef-821c-1c6e74d88494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:20.051967 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:20.051875 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 16:02:19 +0000 UTC" deadline="2028-05-03 09:14:32.269001964 +0000 UTC" Jul 22 16:07:20.051967 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:20.051909 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15617h7m12.217098782s" Jul 22 16:07:20.122838 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:20.122804 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:20.122999 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:20.122923 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:20.349502 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:20.349467 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod313873ed_89d3_43ad_b540_633cb5cbe79a.slice/crio-8205ddcbc25f15ba4427eac2d38026ef8d2a65e29cb3c4bcf411a1e6a59c1cae WatchSource:0}: Error finding container 8205ddcbc25f15ba4427eac2d38026ef8d2a65e29cb3c4bcf411a1e6a59c1cae: Status 404 returned error can't find the container with id 8205ddcbc25f15ba4427eac2d38026ef8d2a65e29cb3c4bcf411a1e6a59c1cae Jul 22 16:07:20.351519 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:20.351382 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd64c7c2f_b2b4_4738_88dd_e56aeeadd228.slice/crio-16ad9097966261810019e49e0755136dd4a8023eaa274ec7a0243db50cdaa524 WatchSource:0}: Error finding container 16ad9097966261810019e49e0755136dd4a8023eaa274ec7a0243db50cdaa524: Status 404 returned error can't find the container with id 16ad9097966261810019e49e0755136dd4a8023eaa274ec7a0243db50cdaa524 Jul 22 16:07:20.593440 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:20.593227 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:20.593536 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:20.593375 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:20.593536 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:20.593504 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs podName:02186816-0aad-4a17-943b-e09d6dca7a03 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:22.593489441 +0000 UTC m=+5.989031254 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs") pod "network-metrics-daemon-lfc4g" (UID: "02186816-0aad-4a17-943b-e09d6dca7a03") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:20.694147 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:20.694114 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:20.694295 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:20.694276 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:20.694354 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:20.694338 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret podName:7ce617c4-ea42-4943-8dde-54b9bfc24d0d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:22.69432167 +0000 UTC m=+6.089863482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret") pod "global-pull-secret-syncer-7zwc4" (UID: "7ce617c4-ea42-4943-8dde-54b9bfc24d0d") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:20.794643 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:20.794594 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4rj\" (UniqueName: \"kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj\") pod \"network-check-target-5sllq\" (UID: \"3e191455-0b73-49ef-821c-1c6e74d88494\") " pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:20.794842 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:20.794817 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:07:20.794972 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:20.794848 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:07:20.794972 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:20.794863 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6g4rj for pod openshift-network-diagnostics/network-check-target-5sllq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:20.794972 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:20.794936 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj podName:3e191455-0b73-49ef-821c-1c6e74d88494 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:22.794913987 +0000 UTC m=+6.190455820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g4rj" (UniqueName: "kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj") pod "network-check-target-5sllq" (UID: "3e191455-0b73-49ef-821c-1c6e74d88494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:21.052388 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.052261 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 16:02:19 +0000 UTC" deadline="2028-04-03 11:30:30.453263904 +0000 UTC" Jul 22 16:07:21.052388 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.052325 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14899h23m9.400943099s" Jul 22 16:07:21.122856 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.122824 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:21.123019 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:21.122968 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:21.123385 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.123364 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:21.123517 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:21.123469 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:21.138751 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.138111 2564 generic.go:356] "Generic (PLEG): container finished" podID="94a4e9989bd9a00f5f1b360ffb81069d" containerID="ff69c1b282c2b794d172dd18424026b5ab2b2d6e3da58e393876c83b6807396f" exitCode=0 Jul 22 16:07:21.138751 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.138189 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" event={"ID":"94a4e9989bd9a00f5f1b360ffb81069d","Type":"ContainerDied","Data":"ff69c1b282c2b794d172dd18424026b5ab2b2d6e3da58e393876c83b6807396f"} Jul 22 16:07:21.149026 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.148963 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hb82z" event={"ID":"e98f2ec9-31ce-45fd-a11f-9bb46626c62b","Type":"ContainerStarted","Data":"51d37b24d946b66f5b5f71039cf1d3d47c31300808100544fec480ad76b49ad5"} Jul 22 16:07:21.152572 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.152544 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" event={"ID":"a7787dd1-9dde-4848-a86b-1076da81f7a0","Type":"ContainerStarted","Data":"9c4220e29bd9591d980721da0bb97c54bddb4b7c107b1957cecde8f1bc4f3f08"} Jul 22 16:07:21.155084 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.155057 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vvbs2" event={"ID":"e89a06fb-1480-471d-be5d-9758ece1a6f8","Type":"ContainerStarted","Data":"c49db9652157f28cacbe0d3107852c7e0856de47fb9dfa2a30ad9241a330d6cc"} Jul 22 16:07:21.161189 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.161105 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" event={"ID":"52990c4f-55be-4b7b-a825-a73f7caf7f08","Type":"ContainerStarted","Data":"5c6d10df1f1c865ed9a095323346197521633e92c2fefe8a7c517d8613b8686a"} Jul 22 16:07:21.165574 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.165504 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4tmkn" event={"ID":"313873ed-89d3-43ad-b540-633cb5cbe79a","Type":"ContainerStarted","Data":"8205ddcbc25f15ba4427eac2d38026ef8d2a65e29cb3c4bcf411a1e6a59c1cae"} Jul 22 16:07:21.181454 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.181427 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal" event={"ID":"a761dd84e540e2dc089b8031677b8ec6","Type":"ContainerStarted","Data":"1983b2f4a41455480b9651c50500944809a314413722fc81d97e4c3987fa33ad"} Jul 22 16:07:21.185557 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.185531 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" event={"ID":"91b8bd81-42f0-4480-9f6f-cdac9397a125","Type":"ContainerStarted","Data":"8ec8f1a1d3a33ef79fd254f09a002a198f1733a9ac899100b3f8f5d9bb040550"} Jul 22 16:07:21.188268 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.188176 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" event={"ID":"b9601bb1-b7f1-4030-82ee-412a348ea931","Type":"ContainerStarted","Data":"118e7a713c15f04d30b874a7deeb074905b72c7be1a4bc036d95237d80f5eea9"} Jul 22 16:07:21.190629 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.190570 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rhlwc" event={"ID":"2d67b9e6-5f4b-421a-af5f-7d8555f9f726","Type":"ContainerStarted","Data":"516bb922a032878c6fdc5a64441d55fcc8f8e7dc73fab50e3a452d5b1209e372"} Jul 22 16:07:21.196789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:21.196668 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nrhcg" event={"ID":"d64c7c2f-b2b4-4738-88dd-e56aeeadd228","Type":"ContainerStarted","Data":"16ad9097966261810019e49e0755136dd4a8023eaa274ec7a0243db50cdaa524"} Jul 22 16:07:22.123417 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:22.122924 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:22.123417 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:22.123058 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:22.220344 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:22.220305 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" event={"ID":"94a4e9989bd9a00f5f1b360ffb81069d","Type":"ContainerStarted","Data":"ec5edbd57ef2ddf687becbcfaab8f8842bfd13e373baa3b79335e98931b661d4"} Jul 22 16:07:22.297539 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:22.297409 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-222.ec2.internal" podStartSLOduration=4.297361409 podStartE2EDuration="4.297361409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:21.200919297 +0000 UTC m=+4.596461131" watchObservedRunningTime="2026-07-22 16:07:22.297361409 +0000 UTC m=+5.692903236" Jul 22 16:07:22.621228 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:22.620575 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:22.621228 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:22.620760 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:22.621228 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:22.620827 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs podName:02186816-0aad-4a17-943b-e09d6dca7a03 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:26.620806782 +0000 UTC m=+10.016348597 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs") pod "network-metrics-daemon-lfc4g" (UID: "02186816-0aad-4a17-943b-e09d6dca7a03") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:22.721277 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:22.721242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:22.721445 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:22.721384 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:22.721445 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:22.721441 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret podName:7ce617c4-ea42-4943-8dde-54b9bfc24d0d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:26.721422014 +0000 UTC m=+10.116963832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret") pod "global-pull-secret-syncer-7zwc4" (UID: "7ce617c4-ea42-4943-8dde-54b9bfc24d0d") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:22.823074 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:22.822198 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4rj\" (UniqueName: \"kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj\") pod \"network-check-target-5sllq\" (UID: \"3e191455-0b73-49ef-821c-1c6e74d88494\") " pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:22.823074 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:22.822382 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:07:22.823074 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:22.822405 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:07:22.823074 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:22.822418 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6g4rj for pod openshift-network-diagnostics/network-check-target-5sllq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:22.823074 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:22.822474 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj podName:3e191455-0b73-49ef-821c-1c6e74d88494 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:26.822454769 +0000 UTC m=+10.217996587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g4rj" (UniqueName: "kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj") pod "network-check-target-5sllq" (UID: "3e191455-0b73-49ef-821c-1c6e74d88494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:23.122520 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:23.122490 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:23.122715 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:23.122669 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:23.123104 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:23.123084 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:23.123233 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:23.123211 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:24.122666 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:24.122447 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:24.122666 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:24.122574 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:25.127180 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:25.126539 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:25.127180 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:25.126669 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:25.127180 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:25.127025 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:25.127180 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:25.127124 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:26.122656 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:26.122146 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:26.122656 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:26.122287 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:26.655308 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:26.655269 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:26.655752 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:26.655445 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:26.655752 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:26.655518 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs podName:02186816-0aad-4a17-943b-e09d6dca7a03 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:34.655496681 +0000 UTC m=+18.051038511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs") pod "network-metrics-daemon-lfc4g" (UID: "02186816-0aad-4a17-943b-e09d6dca7a03") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:26.755929 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:26.755890 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:26.756083 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:26.756053 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:26.756135 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:26.756128 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret podName:7ce617c4-ea42-4943-8dde-54b9bfc24d0d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:34.75610937 +0000 UTC m=+18.151651185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret") pod "global-pull-secret-syncer-7zwc4" (UID: "7ce617c4-ea42-4943-8dde-54b9bfc24d0d") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:26.856982 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:26.856863 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4rj\" (UniqueName: \"kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj\") pod \"network-check-target-5sllq\" (UID: \"3e191455-0b73-49ef-821c-1c6e74d88494\") " pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:26.857148 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:26.857066 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:07:26.857148 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:26.857092 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:07:26.857148 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:26.857105 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6g4rj for pod openshift-network-diagnostics/network-check-target-5sllq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:26.857278 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:26.857166 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj podName:3e191455-0b73-49ef-821c-1c6e74d88494 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:34.857146822 +0000 UTC m=+18.252688657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g4rj" (UniqueName: "kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj") pod "network-check-target-5sllq" (UID: "3e191455-0b73-49ef-821c-1c6e74d88494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:27.123738 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:27.123658 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:27.123897 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:27.123785 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:27.123897 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:27.123850 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:27.124006 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:27.123941 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:28.123073 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:28.122613 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:28.123073 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:28.122752 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:29.123726 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:29.123689 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:29.124159 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:29.123841 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:29.124419 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:29.124399 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:29.124523 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:29.124503 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:30.122543 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:30.122506 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:30.122741 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:30.122638 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:31.123140 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:31.123058 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:31.123526 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:31.123059 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:31.123526 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:31.123214 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:31.123526 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:31.123256 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:32.123028 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:32.122987 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:32.123223 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:32.123124 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:33.122218 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:33.122182 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:33.122394 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:33.122320 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:33.122394 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:33.122377 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:33.122532 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:33.122499 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:34.122783 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:34.122753 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:34.123246 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:34.122853 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:34.716336 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:34.716298 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:34.716521 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:34.716419 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:34.716521 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:34.716487 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs podName:02186816-0aad-4a17-943b-e09d6dca7a03 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:50.716466844 +0000 UTC m=+34.112008703 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs") pod "network-metrics-daemon-lfc4g" (UID: "02186816-0aad-4a17-943b-e09d6dca7a03") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:34.817290 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:34.817242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:34.817472 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:34.817421 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:34.817518 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:34.817505 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret podName:7ce617c4-ea42-4943-8dde-54b9bfc24d0d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:50.817484319 +0000 UTC m=+34.213026133 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret") pod "global-pull-secret-syncer-7zwc4" (UID: "7ce617c4-ea42-4943-8dde-54b9bfc24d0d") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:34.918118 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:34.918078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4rj\" (UniqueName: \"kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj\") pod \"network-check-target-5sllq\" (UID: \"3e191455-0b73-49ef-821c-1c6e74d88494\") " pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:34.918328 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:34.918276 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:07:34.918328 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:34.918304 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:07:34.918328 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:34.918317 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6g4rj for pod openshift-network-diagnostics/network-check-target-5sllq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:34.918476 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:34.918385 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj podName:3e191455-0b73-49ef-821c-1c6e74d88494 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:50.91836622 +0000 UTC m=+34.313908033 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g4rj" (UniqueName: "kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj") pod "network-check-target-5sllq" (UID: "3e191455-0b73-49ef-821c-1c6e74d88494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:35.122779 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:35.122690 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:35.122932 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:35.122839 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:35.122932 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:35.122899 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:35.123370 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:35.123024 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:36.122602 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:36.122567 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:36.122813 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:36.122698 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:37.123348 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:37.123310 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:37.123847 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:37.123438 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:37.123847 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:37.123491 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:37.123847 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:37.123610 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:38.122694 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:38.122656 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:38.122873 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:38.122778 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:39.122921 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.122410 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:39.122921 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:39.122564 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:39.122921 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.122781 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:39.122921 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:39.122891 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:39.253893 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.253647 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nrhcg" event={"ID":"d64c7c2f-b2b4-4738-88dd-e56aeeadd228","Type":"ContainerStarted","Data":"ae5e456002fefeea7bca425844065300e76bd9a1c8428eaff7606fc69baa1f14"} Jul 22 16:07:39.255301 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.255272 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hb82z" event={"ID":"e98f2ec9-31ce-45fd-a11f-9bb46626c62b","Type":"ContainerStarted","Data":"b42754cc0ae534ef5f28fc0d123d600ef254a6d0b323a36c6faa12e44af32f2b"} Jul 22 16:07:39.256810 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.256785 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" event={"ID":"a7787dd1-9dde-4848-a86b-1076da81f7a0","Type":"ContainerStarted","Data":"be2f45661e43175be9ec5f505109ff668e6e59cf5f6df59c74304d04bea4d450"} Jul 22 16:07:39.258342 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.258321 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vvbs2" event={"ID":"e89a06fb-1480-471d-be5d-9758ece1a6f8","Type":"ContainerStarted","Data":"3127df5e82f9f2c06c85b482b78284b15c91bbda016d1b02a5262754e77ea947"} Jul 22 16:07:39.259900 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.259880 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" event={"ID":"52990c4f-55be-4b7b-a825-a73f7caf7f08","Type":"ContainerStarted","Data":"7fa020346c3961878fc6464d5b933463c12e672dfb817de71da9df917b5d4f59"} Jul 22 16:07:39.261037 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.261019 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4tmkn" event={"ID":"313873ed-89d3-43ad-b540-633cb5cbe79a","Type":"ContainerStarted","Data":"2535d6b072adf7902a4073bdd2002b2cafafd662c5175fe14620ab989d0ad9d6"} Jul 22 16:07:39.262419 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.262402 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" event={"ID":"91b8bd81-42f0-4480-9f6f-cdac9397a125","Type":"ContainerStarted","Data":"577293e17b69d7e011b430f57602723612d9bf0eb7cca110cf89b8169a09d45b"} Jul 22 16:07:39.272415 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.270882 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nrhcg" podStartSLOduration=11.801760193 podStartE2EDuration="22.270864961s" totalImagesPullingTime="10.469104768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:39.269273764 +0000 UTC m=+22.664815597" watchObservedRunningTime="2026-07-22 16:07:39.270864961 +0000 UTC m=+22.666406794" Jul 22 16:07:39.272415 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.270997 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-222.ec2.internal" podStartSLOduration=21.270988067 podStartE2EDuration="21.270988067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:22.298407046 +0000 UTC m=+5.693948883" watchObservedRunningTime="2026-07-22 16:07:39.270988067 +0000 UTC m=+22.666529902" Jul 22 16:07:39.290909 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.290852 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4tmkn" podStartSLOduration=3.626126174 podStartE2EDuration="22.290837088s" totalImagesPullingTime="18.664710914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:39.29047767 +0000 UTC m=+22.686019502" watchObservedRunningTime="2026-07-22 16:07:39.290837088 +0000 UTC m=+22.686378951" Jul 22 16:07:39.305930 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.305885 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vvbs2" podStartSLOduration=3.673940099 podStartE2EDuration="22.305868785s" totalImagesPullingTime="18.631928686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:39.305265802 +0000 UTC m=+22.700807635" watchObservedRunningTime="2026-07-22 16:07:39.305868785 +0000 UTC m=+22.701410647" Jul 22 16:07:39.349117 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.349069 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sdbnq" podStartSLOduration=3.7174133510000003 podStartE2EDuration="22.349050232s" totalImagesPullingTime="18.631636881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:39.349040828 +0000 UTC m=+22.744582661" watchObservedRunningTime="2026-07-22 16:07:39.349050232 +0000 UTC m=+22.744592064" Jul 22 16:07:39.366574 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:39.366528 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hb82z" podStartSLOduration=3.73865246 podStartE2EDuration="22.366510913s" totalImagesPullingTime="18.627858453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:39.366140186 +0000 UTC m=+22.761682020" watchObservedRunningTime="2026-07-22 16:07:39.366510913 +0000 UTC m=+22.762052745" Jul 22 16:07:40.123013 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.122823 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:40.123789 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:40.123093 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:40.265553 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.265521 2564 generic.go:356] "Generic (PLEG): container finished" podID="91b8bd81-42f0-4480-9f6f-cdac9397a125" containerID="577293e17b69d7e011b430f57602723612d9bf0eb7cca110cf89b8169a09d45b" exitCode=0 Jul 22 16:07:40.265710 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.265609 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" event={"ID":"91b8bd81-42f0-4480-9f6f-cdac9397a125","Type":"ContainerDied","Data":"577293e17b69d7e011b430f57602723612d9bf0eb7cca110cf89b8169a09d45b"} Jul 22 16:07:40.268492 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.268470 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" event={"ID":"b9601bb1-b7f1-4030-82ee-412a348ea931","Type":"ContainerStarted","Data":"7c4abaf9405ea906ed3b93aaa4be1f1e80c2bc36a513893786789b8978608841"} Jul 22 16:07:40.268600 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.268531 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" event={"ID":"b9601bb1-b7f1-4030-82ee-412a348ea931","Type":"ContainerStarted","Data":"9c4c6a01772faa1afe5d14495185bc986970ccaa4559cfb54bc2f2aaea7773e9"} Jul 22 16:07:40.268600 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.268561 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" event={"ID":"b9601bb1-b7f1-4030-82ee-412a348ea931","Type":"ContainerStarted","Data":"3d92d8fd6b7a86ddf04c6fe47b5f5562d722c845a3bde60fd5d680bf34fb709e"} Jul 22 16:07:40.268600 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.268588 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" event={"ID":"b9601bb1-b7f1-4030-82ee-412a348ea931","Type":"ContainerStarted","Data":"9accf43cf05fc56d02a452494023a797420d99ea32ecabb1c63e193c07ce13ef"} Jul 22 16:07:40.268783 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.268613 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" event={"ID":"b9601bb1-b7f1-4030-82ee-412a348ea931","Type":"ContainerStarted","Data":"56d2aa3be1f63b1a8fe3233c56d0334df47c9924058f90fa9c0f1064495c34fc"} Jul 22 16:07:40.623438 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.623394 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 16:07:40.866955 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.866861 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:40.867377 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:40.867363 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:41.044807 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.044685 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T16:07:40.623419175Z","UUID":"bf567595-df6d-4810-8986-c4977d31bdbf","Handler":null,"Name":"","Endpoint":""} Jul 22 16:07:41.046602 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.046579 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 16:07:41.046602 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.046611 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 16:07:41.122398 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.122312 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:41.122584 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:41.122482 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:41.122584 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.122531 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:41.122716 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:41.122657 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:41.274689 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.274654 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" event={"ID":"b9601bb1-b7f1-4030-82ee-412a348ea931","Type":"ContainerStarted","Data":"2611a2440202ef85cf7778948127159432ffff2eb522c2dd4b317fb103a29698"} Jul 22 16:07:41.276759 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.276722 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rhlwc" event={"ID":"2d67b9e6-5f4b-421a-af5f-7d8555f9f726","Type":"ContainerStarted","Data":"2f9b160bc5176c14acb37028df89dfd7329c2fcfefaa05886411586c396f555b"} Jul 22 16:07:41.279981 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.279941 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" event={"ID":"52990c4f-55be-4b7b-a825-a73f7caf7f08","Type":"ContainerStarted","Data":"a3e07f3c3e8e12a155a3a73bb778bcd60bfd8c5f3026928a517d85bff43d4c67"} Jul 22 16:07:41.280238 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.280212 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:41.281054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.281027 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vvbs2" Jul 22 16:07:41.318957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:41.318896 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rhlwc" podStartSLOduration=5.684674423 podStartE2EDuration="24.318872378s" totalImagesPullingTime="18.634197955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:41.318605867 +0000 UTC m=+24.714147700" watchObservedRunningTime="2026-07-22 16:07:41.318872378 +0000 UTC m=+24.714414215" Jul 22 16:07:42.122167 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:42.122133 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:42.122389 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:42.122235 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:42.285062 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:42.285021 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" event={"ID":"52990c4f-55be-4b7b-a825-a73f7caf7f08","Type":"ContainerStarted","Data":"a1ce4137e65409fa4df86518de535bddef5fbc87b6ac5b4f2ea61456778a5ffa"} Jul 22 16:07:42.305927 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:42.305864 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9z798" podStartSLOduration=4.417625158 podStartE2EDuration="25.305844237s" totalImagesPullingTime="20.888219079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:42.305237877 +0000 UTC m=+25.700779713" watchObservedRunningTime="2026-07-22 16:07:42.305844237 +0000 UTC m=+25.701386069" Jul 22 16:07:43.122968 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:43.122934 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:43.123163 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:43.122934 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:43.123163 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:43.123093 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:43.123163 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:43.123103 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:43.291155 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:43.290900 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" event={"ID":"b9601bb1-b7f1-4030-82ee-412a348ea931","Type":"ContainerStarted","Data":"baf021cc190a79e64ecd3842c8fc005bebea74e67d5321a39bb3758b17b525fd"} Jul 22 16:07:44.122583 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:44.122547 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:44.122787 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:44.122697 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:45.122838 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:45.122661 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:45.123643 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:45.122662 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:45.123643 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:45.122932 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:45.123643 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:45.122980 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:45.300144 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:45.297699 2564 generic.go:356] "Generic (PLEG): container finished" podID="91b8bd81-42f0-4480-9f6f-cdac9397a125" containerID="ccd16d614242b4f8a847e02cfdb23bfdcf9e3e2ccee2dcf7a9b98d0c6c01f52e" exitCode=0 Jul 22 16:07:45.300144 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:45.297805 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" event={"ID":"91b8bd81-42f0-4480-9f6f-cdac9397a125","Type":"ContainerDied","Data":"ccd16d614242b4f8a847e02cfdb23bfdcf9e3e2ccee2dcf7a9b98d0c6c01f52e"} Jul 22 16:07:45.303844 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:45.303814 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" event={"ID":"b9601bb1-b7f1-4030-82ee-412a348ea931","Type":"ContainerStarted","Data":"30c64ca4c9100e5a24c880b59516a90b0cb199ae620586ba8612c5a72c0b7092"} Jul 22 16:07:45.304117 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:45.304094 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:45.319604 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:45.319581 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:45.355665 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:45.355587 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" podStartSLOduration=9.221716634 podStartE2EDuration="28.355569693s" totalImagesPullingTime="19.133853059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:45.353981977 +0000 UTC m=+28.749523810" watchObservedRunningTime="2026-07-22 16:07:45.355569693 +0000 UTC m=+28.751111531" Jul 22 16:07:46.122122 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:46.122097 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:46.122351 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:46.122191 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:46.306292 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:46.306262 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:46.306757 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:46.306345 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:46.320854 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:46.320828 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:07:46.773186 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:46.773109 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lfc4g"] Jul 22 16:07:46.773323 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:46.773273 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:46.773391 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:46.773374 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:46.776068 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:46.776043 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5sllq"] Jul 22 16:07:46.776195 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:46.776153 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:46.776271 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:46.776250 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:46.788424 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:46.788396 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7zwc4"] Jul 22 16:07:46.788537 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:46.788484 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:46.788593 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:46.788571 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:47.309698 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:47.309660 2564 generic.go:356] "Generic (PLEG): container finished" podID="91b8bd81-42f0-4480-9f6f-cdac9397a125" containerID="427adc2fad1adaafc7ada65f72f49b33cdc190ec5dc7e924a7bc5a002c3021d2" exitCode=0 Jul 22 16:07:47.310080 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:47.309708 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" event={"ID":"91b8bd81-42f0-4480-9f6f-cdac9397a125","Type":"ContainerDied","Data":"427adc2fad1adaafc7ada65f72f49b33cdc190ec5dc7e924a7bc5a002c3021d2"} Jul 22 16:07:48.122687 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:48.122651 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:48.122852 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:48.122650 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:48.122852 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:48.122766 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:48.122926 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:48.122866 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:49.122842 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:49.122745 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:49.123303 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:49.122868 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:49.316846 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:49.316808 2564 generic.go:356] "Generic (PLEG): container finished" podID="91b8bd81-42f0-4480-9f6f-cdac9397a125" containerID="adb85ca3c33d22f320b6a9193d863a7bd12c69445628b82db23fb5b9c44421dd" exitCode=0 Jul 22 16:07:49.317021 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:49.316869 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" event={"ID":"91b8bd81-42f0-4480-9f6f-cdac9397a125","Type":"ContainerDied","Data":"adb85ca3c33d22f320b6a9193d863a7bd12c69445628b82db23fb5b9c44421dd"} Jul 22 16:07:50.122873 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:50.122830 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:50.123395 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:50.122883 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:50.123395 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:50.122976 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:50.123395 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:50.123106 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:50.735124 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:50.735084 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:50.735316 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:50.735248 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:50.735361 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:50.735323 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs podName:02186816-0aad-4a17-943b-e09d6dca7a03 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:22.735305152 +0000 UTC m=+66.130846971 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs") pod "network-metrics-daemon-lfc4g" (UID: "02186816-0aad-4a17-943b-e09d6dca7a03") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:07:50.835914 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:50.835874 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:50.836070 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:50.836045 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:50.836125 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:50.836112 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret podName:7ce617c4-ea42-4943-8dde-54b9bfc24d0d nodeName:}" failed. No retries permitted until 2026-07-22 16:08:22.836092773 +0000 UTC m=+66.231634584 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret") pod "global-pull-secret-syncer-7zwc4" (UID: "7ce617c4-ea42-4943-8dde-54b9bfc24d0d") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:07:50.936892 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:50.936856 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4rj\" (UniqueName: \"kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj\") pod \"network-check-target-5sllq\" (UID: \"3e191455-0b73-49ef-821c-1c6e74d88494\") " pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:50.937074 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:50.936990 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:07:50.937074 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:50.937008 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:07:50.937074 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:50.937016 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6g4rj for pod openshift-network-diagnostics/network-check-target-5sllq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:50.937074 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:50.937071 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj podName:3e191455-0b73-49ef-821c-1c6e74d88494 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:22.937053078 +0000 UTC m=+66.332594909 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6g4rj" (UniqueName: "kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj") pod "network-check-target-5sllq" (UID: "3e191455-0b73-49ef-821c-1c6e74d88494") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:07:51.123057 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:51.122979 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:51.123500 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:51.123120 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lfc4g" podUID="02186816-0aad-4a17-943b-e09d6dca7a03" Jul 22 16:07:52.123016 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:52.122981 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:52.123214 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:52.122987 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:52.123214 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:52.123099 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zwc4" podUID="7ce617c4-ea42-4943-8dde-54b9bfc24d0d" Jul 22 16:07:52.123214 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:52.123154 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sllq" podUID="3e191455-0b73-49ef-821c-1c6e74d88494" Jul 22 16:07:52.923031 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:52.922955 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-222.ec2.internal" event="NodeReady" Jul 22 16:07:52.923177 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:52.923108 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 16:07:52.972933 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:52.972900 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bb85c9569-5kqjz"] Jul 22 16:07:53.006692 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.006659 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nhdrz"] Jul 22 16:07:53.006879 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.006753 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.010079 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.010052 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 16:07:53.010288 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.010268 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 16:07:53.010424 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.010306 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-crmjn\"" Jul 22 16:07:53.010588 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.010571 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 16:07:53.015835 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.015813 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 16:07:53.020565 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.020544 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w4dkw"] Jul 22 16:07:53.020705 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.020690 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.023146 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.023127 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 16:07:53.023252 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.023195 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 16:07:53.023252 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.023202 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wdbjp\"" Jul 22 16:07:53.034444 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.034422 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w4dkw"] Jul 22 16:07:53.034554 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.034463 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nhdrz"] Jul 22 16:07:53.034554 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.034491 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bb85c9569-5kqjz"] Jul 22 16:07:53.034554 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.034530 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w4dkw" Jul 22 16:07:53.037234 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.037214 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 16:07:53.037641 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.037610 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7zs9d\"" Jul 22 16:07:53.037735 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.037697 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 16:07:53.037877 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.037860 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 16:07:53.127189 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.127157 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:07:53.130391 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.130365 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 16:07:53.130547 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.130392 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rb5n6\"" Jul 22 16:07:53.153962 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.153933 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-image-registry-private-configuration\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.154108 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.153967 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-trusted-ca\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.154108 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.153996 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58b9ab02-b98e-466f-b11c-2600c9e47261-cert\") pod \"ingress-canary-w4dkw\" (UID: \"58b9ab02-b98e-466f-b11c-2600c9e47261\") " pod="openshift-ingress-canary/ingress-canary-w4dkw" Jul 22 16:07:53.154108 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154017 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-installation-pull-secrets\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.154108 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154049 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.154108 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154078 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqwcx\" (UniqueName: \"kubernetes.io/projected/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-kube-api-access-dqwcx\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.154256 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154113 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.154256 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154128 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be2d0bc0-a54e-4508-9bd3-08facb2435d5-ca-trust-extracted\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.154256 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-bound-sa-token\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.154256 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154193 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-certificates\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.154368 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154249 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-tmp-dir\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.154368 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154277 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzb6h\" (UniqueName: \"kubernetes.io/projected/58b9ab02-b98e-466f-b11c-2600c9e47261-kube-api-access-zzb6h\") pod \"ingress-canary-w4dkw\" (UID: \"58b9ab02-b98e-466f-b11c-2600c9e47261\") " pod="openshift-ingress-canary/ingress-canary-w4dkw" Jul 22 16:07:53.154368 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154315 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-config-volume\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.154368 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.154335 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4txh\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-kube-api-access-s4txh\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.255279 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255241 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-tmp-dir\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.255279 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255281 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzb6h\" (UniqueName: \"kubernetes.io/projected/58b9ab02-b98e-466f-b11c-2600c9e47261-kube-api-access-zzb6h\") pod \"ingress-canary-w4dkw\" (UID: \"58b9ab02-b98e-466f-b11c-2600c9e47261\") " pod="openshift-ingress-canary/ingress-canary-w4dkw" Jul 22 16:07:53.255501 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255303 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-config-volume\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.255501 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255328 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4txh\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-kube-api-access-s4txh\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.255501 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255382 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-image-registry-private-configuration\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.255501 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-trusted-ca\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.255501 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255441 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58b9ab02-b98e-466f-b11c-2600c9e47261-cert\") pod \"ingress-canary-w4dkw\" (UID: \"58b9ab02-b98e-466f-b11c-2600c9e47261\") " pod="openshift-ingress-canary/ingress-canary-w4dkw" Jul 22 16:07:53.255501 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255463 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-installation-pull-secrets\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.255827 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255690 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-tmp-dir\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.255827 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255780 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.255827 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255823 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqwcx\" (UniqueName: \"kubernetes.io/projected/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-kube-api-access-dqwcx\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.255963 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255852 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.255963 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255874 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be2d0bc0-a54e-4508-9bd3-08facb2435d5-ca-trust-extracted\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.255963 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255895 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-bound-sa-token\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.255963 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.255923 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-certificates\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.256147 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:53.256051 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:07:53.256147 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:53.256069 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bb85c9569-5kqjz: secret "image-registry-tls" not found Jul 22 16:07:53.256236 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:53.256152 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls podName:be2d0bc0-a54e-4508-9bd3-08facb2435d5 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:53.756126499 +0000 UTC m=+37.151668310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls") pod "image-registry-6bb85c9569-5kqjz" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5") : secret "image-registry-tls" not found Jul 22 16:07:53.256236 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.256168 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-config-volume\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.256507 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.256350 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-trusted-ca\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.256507 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.256415 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-certificates\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.256507 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:53.256444 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:07:53.256507 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.256487 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be2d0bc0-a54e-4508-9bd3-08facb2435d5-ca-trust-extracted\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.256507 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:53.256506 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls podName:7282bf8f-d9c6-4f64-8e4f-ab36bef8369d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:53.756487752 +0000 UTC m=+37.152029566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls") pod "dns-default-nhdrz" (UID: "7282bf8f-d9c6-4f64-8e4f-ab36bef8369d") : secret "dns-default-metrics-tls" not found Jul 22 16:07:53.259522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.259497 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-image-registry-private-configuration\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.259522 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.259512 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-installation-pull-secrets\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.259732 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.259714 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58b9ab02-b98e-466f-b11c-2600c9e47261-cert\") pod \"ingress-canary-w4dkw\" (UID: \"58b9ab02-b98e-466f-b11c-2600c9e47261\") " pod="openshift-ingress-canary/ingress-canary-w4dkw" Jul 22 16:07:53.264799 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.264774 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-bound-sa-token\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.265193 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.265160 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqwcx\" (UniqueName: \"kubernetes.io/projected/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-kube-api-access-dqwcx\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.265314 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.265251 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzb6h\" (UniqueName: \"kubernetes.io/projected/58b9ab02-b98e-466f-b11c-2600c9e47261-kube-api-access-zzb6h\") pod \"ingress-canary-w4dkw\" (UID: \"58b9ab02-b98e-466f-b11c-2600c9e47261\") " pod="openshift-ingress-canary/ingress-canary-w4dkw" Jul 22 16:07:53.265653 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.265635 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4txh\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-kube-api-access-s4txh\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.343547 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.343503 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w4dkw" Jul 22 16:07:53.529408 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.529315 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w4dkw"] Jul 22 16:07:53.532238 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:07:53.532210 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58b9ab02_b98e_466f_b11c_2600c9e47261.slice/crio-d8d21fe686fa48c46105edc2d6f41d88d7983fde8f2e088109db7f0f626347b4 WatchSource:0}: Error finding container d8d21fe686fa48c46105edc2d6f41d88d7983fde8f2e088109db7f0f626347b4: Status 404 returned error can't find the container with id d8d21fe686fa48c46105edc2d6f41d88d7983fde8f2e088109db7f0f626347b4 Jul 22 16:07:53.760160 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.760129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:53.760268 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:53.760188 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:53.760303 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:53.760274 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:07:53.760344 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:53.760324 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:07:53.760344 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:53.760338 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls podName:7282bf8f-d9c6-4f64-8e4f-ab36bef8369d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:54.76032354 +0000 UTC m=+38.155865351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls") pod "dns-default-nhdrz" (UID: "7282bf8f-d9c6-4f64-8e4f-ab36bef8369d") : secret "dns-default-metrics-tls" not found Jul 22 16:07:53.760344 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:53.760340 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bb85c9569-5kqjz: secret "image-registry-tls" not found Jul 22 16:07:53.760438 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:53.760381 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls podName:be2d0bc0-a54e-4508-9bd3-08facb2435d5 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:54.760369536 +0000 UTC m=+38.155911347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls") pod "image-registry-6bb85c9569-5kqjz" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5") : secret "image-registry-tls" not found Jul 22 16:07:54.122268 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.122182 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:07:54.122437 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.122182 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:07:54.126501 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.126469 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s68m6\"" Jul 22 16:07:54.126650 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.126475 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 16:07:54.127663 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.127641 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 16:07:54.128047 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.127648 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 16:07:54.330335 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.330286 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w4dkw" event={"ID":"58b9ab02-b98e-466f-b11c-2600c9e47261","Type":"ContainerStarted","Data":"d8d21fe686fa48c46105edc2d6f41d88d7983fde8f2e088109db7f0f626347b4"} Jul 22 16:07:54.333321 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.333286 2564 generic.go:356] "Generic (PLEG): container finished" podID="91b8bd81-42f0-4480-9f6f-cdac9397a125" containerID="a3da51f57c2be0d75631912ee87321dcc4df385b6b7188101760501cbd558901" exitCode=0 Jul 22 16:07:54.333477 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.333330 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" event={"ID":"91b8bd81-42f0-4480-9f6f-cdac9397a125","Type":"ContainerDied","Data":"a3da51f57c2be0d75631912ee87321dcc4df385b6b7188101760501cbd558901"} Jul 22 16:07:54.769344 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.769308 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:54.769533 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:54.769366 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:54.769533 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:54.769467 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:07:54.769656 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:54.769533 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:07:54.769656 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:54.769541 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls podName:7282bf8f-d9c6-4f64-8e4f-ab36bef8369d nodeName:}" failed. No retries permitted until 2026-07-22 16:07:56.769521489 +0000 UTC m=+40.165063302 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls") pod "dns-default-nhdrz" (UID: "7282bf8f-d9c6-4f64-8e4f-ab36bef8369d") : secret "dns-default-metrics-tls" not found Jul 22 16:07:54.769656 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:54.769548 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bb85c9569-5kqjz: secret "image-registry-tls" not found Jul 22 16:07:54.769656 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:54.769591 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls podName:be2d0bc0-a54e-4508-9bd3-08facb2435d5 nodeName:}" failed. No retries permitted until 2026-07-22 16:07:56.769580084 +0000 UTC m=+40.165121899 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls") pod "image-registry-6bb85c9569-5kqjz" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5") : secret "image-registry-tls" not found Jul 22 16:07:55.338504 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:55.338467 2564 generic.go:356] "Generic (PLEG): container finished" podID="91b8bd81-42f0-4480-9f6f-cdac9397a125" containerID="19855db9fcdcde366116773cbf098d604e205faf984d687db5b303f4427893a6" exitCode=0 Jul 22 16:07:55.338897 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:55.338520 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" event={"ID":"91b8bd81-42f0-4480-9f6f-cdac9397a125","Type":"ContainerDied","Data":"19855db9fcdcde366116773cbf098d604e205faf984d687db5b303f4427893a6"} Jul 22 16:07:56.345902 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:56.345707 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" event={"ID":"91b8bd81-42f0-4480-9f6f-cdac9397a125","Type":"ContainerStarted","Data":"c9213be4c02a8636fd8799307ab409f7079793532c8abcc81419af2224ea2aac"} Jul 22 16:07:56.347221 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:56.347189 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w4dkw" event={"ID":"58b9ab02-b98e-466f-b11c-2600c9e47261","Type":"ContainerStarted","Data":"349aa741ddd2be87597abe62ae6847ff70228c46b54abec2c9e6d4de075bfcba"} Jul 22 16:07:56.378482 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:56.378429 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p8jnb" podStartSLOduration=10.016168343 podStartE2EDuration="39.378404321s" totalImagesPullingTime="29.362235978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:56.377958458 +0000 UTC m=+39.773500291" watchObservedRunningTime="2026-07-22 16:07:56.378404321 +0000 UTC m=+39.773946153" Jul 22 16:07:56.397383 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:56.397330 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w4dkw" podStartSLOduration=1.847930163 podStartE2EDuration="4.397313235s" totalImagesPullingTime="2.549383072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:07:56.396595698 +0000 UTC m=+39.792137532" watchObservedRunningTime="2026-07-22 16:07:56.397313235 +0000 UTC m=+39.792855067" Jul 22 16:07:56.784008 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:56.783958 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:07:56.784008 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:07:56.784017 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:07:56.784216 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:56.784109 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:07:56.784216 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:56.784128 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:07:56.784216 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:56.784139 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bb85c9569-5kqjz: secret "image-registry-tls" not found Jul 22 16:07:56.784216 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:56.784171 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls podName:7282bf8f-d9c6-4f64-8e4f-ab36bef8369d nodeName:}" failed. No retries permitted until 2026-07-22 16:08:00.78415485 +0000 UTC m=+44.179696661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls") pod "dns-default-nhdrz" (UID: "7282bf8f-d9c6-4f64-8e4f-ab36bef8369d") : secret "dns-default-metrics-tls" not found Jul 22 16:07:56.784216 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:07:56.784187 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls podName:be2d0bc0-a54e-4508-9bd3-08facb2435d5 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:00.784176957 +0000 UTC m=+44.179718768 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls") pod "image-registry-6bb85c9569-5kqjz" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5") : secret "image-registry-tls" not found Jul 22 16:08:00.815815 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:00.815770 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:08:00.815815 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:00.815818 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:08:00.816245 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:00.815922 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:08:00.816245 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:00.815933 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:08:00.816245 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:00.815944 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bb85c9569-5kqjz: secret "image-registry-tls" not found Jul 22 16:08:00.816245 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:00.815984 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls podName:7282bf8f-d9c6-4f64-8e4f-ab36bef8369d nodeName:}" failed. No retries permitted until 2026-07-22 16:08:08.815968224 +0000 UTC m=+52.211510038 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls") pod "dns-default-nhdrz" (UID: "7282bf8f-d9c6-4f64-8e4f-ab36bef8369d") : secret "dns-default-metrics-tls" not found Jul 22 16:08:00.816245 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:00.815997 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls podName:be2d0bc0-a54e-4508-9bd3-08facb2435d5 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:08.815991764 +0000 UTC m=+52.211533575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls") pod "image-registry-6bb85c9569-5kqjz" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5") : secret "image-registry-tls" not found Jul 22 16:08:01.147681 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.147592 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-j4g57"] Jul 22 16:08:01.178981 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.178951 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j4g57"] Jul 22 16:08:01.179120 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.179069 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.184245 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.184218 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 16:08:01.184403 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.184244 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 16:08:01.184403 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.184244 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 16:08:01.185398 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.185375 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cvcbr\"" Jul 22 16:08:01.185398 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.185386 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 16:08:01.219896 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.219859 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.220044 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.219926 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-crio-socket\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.220044 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.219969 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-data-volume\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.220044 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.219996 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.220044 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.220015 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slknw\" (UniqueName: \"kubernetes.io/projected/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-api-access-slknw\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.320755 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.320712 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slknw\" (UniqueName: \"kubernetes.io/projected/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-api-access-slknw\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.320904 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.320808 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.320904 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.320885 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-crio-socket\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.320998 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.320918 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-data-volume\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.320998 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.320941 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.320998 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:01.320964 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:08:01.321104 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:01.321011 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:08:01.321104 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:01.321044 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls podName:201d6928-9fae-43b8-b0a4-d1ce64c9d9c7 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:01.821022417 +0000 UTC m=+45.216564230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j4g57" (UID: "201d6928-9fae-43b8-b0a4-d1ce64c9d9c7") : secret "insights-runtime-extractor-tls" not found Jul 22 16:08:01.321104 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:01.321067 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm podName:201d6928-9fae-43b8-b0a4-d1ce64c9d9c7 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:01.821057631 +0000 UTC m=+45.216599442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j4g57" (UID: "201d6928-9fae-43b8-b0a4-d1ce64c9d9c7") : configmap "kube-rbac-proxy" not found Jul 22 16:08:01.321104 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.321093 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-crio-socket\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.321301 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.321285 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-data-volume\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.332387 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.332362 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slknw\" (UniqueName: \"kubernetes.io/projected/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-api-access-slknw\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.823421 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.823379 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.823842 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:01.823462 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:01.823842 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:01.823520 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:08:01.823842 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:01.823584 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls podName:201d6928-9fae-43b8-b0a4-d1ce64c9d9c7 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:02.823570039 +0000 UTC m=+46.219111851 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j4g57" (UID: "201d6928-9fae-43b8-b0a4-d1ce64c9d9c7") : secret "insights-runtime-extractor-tls" not found Jul 22 16:08:01.823842 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:01.823528 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:08:01.823842 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:01.823692 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm podName:201d6928-9fae-43b8-b0a4-d1ce64c9d9c7 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:02.823677525 +0000 UTC m=+46.219219337 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j4g57" (UID: "201d6928-9fae-43b8-b0a4-d1ce64c9d9c7") : configmap "kube-rbac-proxy" not found Jul 22 16:08:02.829238 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:02.829193 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:02.829737 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:02.829257 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:02.829737 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:02.829310 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:08:02.829737 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:02.829367 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:08:02.829737 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:02.829378 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm podName:201d6928-9fae-43b8-b0a4-d1ce64c9d9c7 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:04.829363477 +0000 UTC m=+48.224905288 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j4g57" (UID: "201d6928-9fae-43b8-b0a4-d1ce64c9d9c7") : configmap "kube-rbac-proxy" not found Jul 22 16:08:02.829737 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:02.829405 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls podName:201d6928-9fae-43b8-b0a4-d1ce64c9d9c7 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:04.829394424 +0000 UTC m=+48.224936235 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j4g57" (UID: "201d6928-9fae-43b8-b0a4-d1ce64c9d9c7") : secret "insights-runtime-extractor-tls" not found Jul 22 16:08:03.008776 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:03.008747 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hb82z_e98f2ec9-31ce-45fd-a11f-9bb46626c62b/dns-node-resolver/0.log" Jul 22 16:08:04.208238 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:04.208200 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nrhcg_d64c7c2f-b2b4-4738-88dd-e56aeeadd228/node-ca/0.log" Jul 22 16:08:04.844963 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:04.844917 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:04.845149 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:04.845005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:04.845149 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:04.845061 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:08:04.845149 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:04.845130 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls podName:201d6928-9fae-43b8-b0a4-d1ce64c9d9c7 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:08.845114129 +0000 UTC m=+52.240655940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j4g57" (UID: "201d6928-9fae-43b8-b0a4-d1ce64c9d9c7") : secret "insights-runtime-extractor-tls" not found Jul 22 16:08:04.845149 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:04.845070 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:08:04.845300 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:04.845165 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm podName:201d6928-9fae-43b8-b0a4-d1ce64c9d9c7 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:08.84515821 +0000 UTC m=+52.240700022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j4g57" (UID: "201d6928-9fae-43b8-b0a4-d1ce64c9d9c7") : configmap "kube-rbac-proxy" not found Jul 22 16:08:05.209889 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:05.209818 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w4dkw_58b9ab02-b98e-466f-b11c-2600c9e47261/serve-healthcheck-canary/0.log" Jul 22 16:08:08.874962 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:08.874927 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:08.875359 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:08.874970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:08:08.875359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:08.875087 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:08:08.875359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:08.875151 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls podName:201d6928-9fae-43b8-b0a4-d1ce64c9d9c7 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:16.875134676 +0000 UTC m=+60.270676487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j4g57" (UID: "201d6928-9fae-43b8-b0a4-d1ce64c9d9c7") : secret "insights-runtime-extractor-tls" not found Jul 22 16:08:08.875359 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:08.875093 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:08:08.875359 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:08.875214 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:08.875359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:08.875161 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:08:08.875359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:08.875245 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bb85c9569-5kqjz: secret "image-registry-tls" not found Jul 22 16:08:08.875359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:08.875280 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls podName:be2d0bc0-a54e-4508-9bd3-08facb2435d5 nodeName:}" failed. No retries permitted until 2026-07-22 16:08:24.875269232 +0000 UTC m=+68.270811043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls") pod "image-registry-6bb85c9569-5kqjz" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5") : secret "image-registry-tls" not found Jul 22 16:08:08.875359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:08.875195 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:08:08.875359 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:08.875311 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls podName:7282bf8f-d9c6-4f64-8e4f-ab36bef8369d nodeName:}" failed. No retries permitted until 2026-07-22 16:08:24.875304007 +0000 UTC m=+68.270845818 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls") pod "dns-default-nhdrz" (UID: "7282bf8f-d9c6-4f64-8e4f-ab36bef8369d") : secret "dns-default-metrics-tls" not found Jul 22 16:08:08.875726 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:08.875710 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:16.938788 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:16.938726 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:16.941077 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:16.941048 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/201d6928-9fae-43b8-b0a4-d1ce64c9d9c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4g57\" (UID: \"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7\") " pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:17.092242 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:17.092206 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cvcbr\"" Jul 22 16:08:17.099236 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:17.099208 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j4g57" Jul 22 16:08:17.243979 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:17.243949 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j4g57"] Jul 22 16:08:17.397070 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:17.397037 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j4g57" event={"ID":"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7","Type":"ContainerStarted","Data":"557549d0d254efd6f42b79078b9255799039c701c88beb5d93b4e4f6895e97af"} Jul 22 16:08:18.323888 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:18.323857 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lsfmv" Jul 22 16:08:22.411146 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.411060 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j4g57" event={"ID":"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7","Type":"ContainerStarted","Data":"85ab68d90da3407a669fa5364222bf7a8e490e5118d08c57e04f29f8bcd0abde"} Jul 22 16:08:22.783341 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.783293 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:08:22.787068 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.787040 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 16:08:22.795820 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.795784 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02186816-0aad-4a17-943b-e09d6dca7a03-metrics-certs\") pod \"network-metrics-daemon-lfc4g\" (UID: \"02186816-0aad-4a17-943b-e09d6dca7a03\") " pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:08:22.843557 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.843524 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rb5n6\"" Jul 22 16:08:22.851057 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.851030 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lfc4g" Jul 22 16:08:22.884718 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.884679 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:08:22.888050 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.888024 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 16:08:22.898299 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.898232 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ce617c4-ea42-4943-8dde-54b9bfc24d0d-original-pull-secret\") pod \"global-pull-secret-syncer-7zwc4\" (UID: \"7ce617c4-ea42-4943-8dde-54b9bfc24d0d\") " pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:08:22.936520 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.936452 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zwc4" Jul 22 16:08:22.986247 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.985764 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g4rj\" (UniqueName: \"kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj\") pod \"network-check-target-5sllq\" (UID: \"3e191455-0b73-49ef-821c-1c6e74d88494\") " pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:08:22.989322 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:22.989241 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 16:08:23.004469 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:23.000488 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 16:08:23.010299 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:23.010237 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g4rj\" (UniqueName: \"kubernetes.io/projected/3e191455-0b73-49ef-821c-1c6e74d88494-kube-api-access-6g4rj\") pod \"network-check-target-5sllq\" (UID: \"3e191455-0b73-49ef-821c-1c6e74d88494\") " pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:08:23.016073 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:23.015901 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lfc4g"] Jul 22 16:08:23.020210 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:23.020183 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02186816_0aad_4a17_943b_e09d6dca7a03.slice/crio-241087f680b74f2f1e63730dd6b612b923811605e42a8e1a50968b88eea0bde6 WatchSource:0}: Error finding container 241087f680b74f2f1e63730dd6b612b923811605e42a8e1a50968b88eea0bde6: Status 404 returned error can't find the container with id 241087f680b74f2f1e63730dd6b612b923811605e42a8e1a50968b88eea0bde6 Jul 22 16:08:23.103610 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:23.103571 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7zwc4"] Jul 22 16:08:23.107833 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:23.107793 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce617c4_ea42_4943_8dde_54b9bfc24d0d.slice/crio-913b06649fb856776a1a2d5750a8ca6dc46e43558e0b5876e57db0a61d3611ba WatchSource:0}: Error finding container 913b06649fb856776a1a2d5750a8ca6dc46e43558e0b5876e57db0a61d3611ba: Status 404 returned error can't find the container with id 913b06649fb856776a1a2d5750a8ca6dc46e43558e0b5876e57db0a61d3611ba Jul 22 16:08:23.234854 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:23.234821 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s68m6\"" Jul 22 16:08:23.242376 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:23.242309 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:08:23.400285 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:23.400242 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5sllq"] Jul 22 16:08:23.404371 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:23.404340 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e191455_0b73_49ef_821c_1c6e74d88494.slice/crio-50f1b049d634e07c2a84e393bbb2176fee359d722c48197e479dffe2c2d2f871 WatchSource:0}: Error finding container 50f1b049d634e07c2a84e393bbb2176fee359d722c48197e479dffe2c2d2f871: Status 404 returned error can't find the container with id 50f1b049d634e07c2a84e393bbb2176fee359d722c48197e479dffe2c2d2f871 Jul 22 16:08:23.415996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:23.415957 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5sllq" event={"ID":"3e191455-0b73-49ef-821c-1c6e74d88494","Type":"ContainerStarted","Data":"50f1b049d634e07c2a84e393bbb2176fee359d722c48197e479dffe2c2d2f871"} Jul 22 16:08:23.417689 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:23.417659 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7zwc4" event={"ID":"7ce617c4-ea42-4943-8dde-54b9bfc24d0d","Type":"ContainerStarted","Data":"913b06649fb856776a1a2d5750a8ca6dc46e43558e0b5876e57db0a61d3611ba"} Jul 22 16:08:23.419232 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:23.419204 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lfc4g" event={"ID":"02186816-0aad-4a17-943b-e09d6dca7a03","Type":"ContainerStarted","Data":"241087f680b74f2f1e63730dd6b612b923811605e42a8e1a50968b88eea0bde6"} Jul 22 16:08:24.425277 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.425198 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lfc4g" event={"ID":"02186816-0aad-4a17-943b-e09d6dca7a03","Type":"ContainerStarted","Data":"e7c15b95b9ba44dcdbc2254aece53591fb7b10e66cffe00e364da8ebd42d70b1"} Jul 22 16:08:24.862779 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.861794 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5"] Jul 22 16:08:24.873175 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.873136 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" Jul 22 16:08:24.877087 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.877059 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 16:08:24.877900 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.877870 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 16:08:24.878681 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.878223 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jjxhx\"" Jul 22 16:08:24.893385 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.893356 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5"] Jul 22 16:08:24.897323 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.896319 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc"] Jul 22 16:08:24.897323 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.896442 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:24.898909 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.898890 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" Jul 22 16:08:24.899638 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.899593 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 16:08:24.899743 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.899701 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 16:08:24.900448 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.900318 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 22 16:08:24.900822 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.900591 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 16:08:24.901124 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.901053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:08:24.901124 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.901098 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:08:24.901356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.901154 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1467496e-62de-4881-b8a9-9e090e9e0c1b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-wgvx5\" (UID: \"1467496e-62de-4881-b8a9-9e090e9e0c1b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" Jul 22 16:08:24.901356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.901183 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1467496e-62de-4881-b8a9-9e090e9e0c1b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-wgvx5\" (UID: \"1467496e-62de-4881-b8a9-9e090e9e0c1b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" Jul 22 16:08:24.901960 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.901919 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 22 16:08:24.902823 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.902801 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-nnc5w\"" Jul 22 16:08:24.905085 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.905045 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") pod \"image-registry-6bb85c9569-5kqjz\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:08:24.909668 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.909594 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7282bf8f-d9c6-4f64-8e4f-ab36bef8369d-metrics-tls\") pod \"dns-default-nhdrz\" (UID: \"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d\") " pod="openshift-dns/dns-default-nhdrz" Jul 22 16:08:24.913701 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.913666 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5"] Jul 22 16:08:24.925148 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.925087 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc"] Jul 22 16:08:24.951101 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.951065 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bb85c9569-5kqjz"] Jul 22 16:08:24.954057 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.954029 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-crmjn\"" Jul 22 16:08:24.962318 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.962037 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:08:24.966263 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:24.965958 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5"] Jul 22 16:08:25.002058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.002021 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1467496e-62de-4881-b8a9-9e090e9e0c1b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-wgvx5\" (UID: \"1467496e-62de-4881-b8a9-9e090e9e0c1b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" Jul 22 16:08:25.002252 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.002070 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d94a6f3-2731-4fc1-9d03-fb98f2f37092-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc\" (UID: \"7d94a6f3-2731-4fc1-9d03-fb98f2f37092\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" Jul 22 16:08:25.002252 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.002101 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1467496e-62de-4881-b8a9-9e090e9e0c1b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-wgvx5\" (UID: \"1467496e-62de-4881-b8a9-9e090e9e0c1b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" Jul 22 16:08:25.002252 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.002125 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22745\" (UniqueName: \"kubernetes.io/projected/7d94a6f3-2731-4fc1-9d03-fb98f2f37092-kube-api-access-22745\") pod \"managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc\" (UID: \"7d94a6f3-2731-4fc1-9d03-fb98f2f37092\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" Jul 22 16:08:25.002252 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.002147 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d41ac46-970e-446b-a807-4030fe58e165-tmp\") pod \"klusterlet-addon-workmgr-5657cdcbcb-hfmw5\" (UID: \"7d41ac46-970e-446b-a807-4030fe58e165\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:25.002252 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.002191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7d41ac46-970e-446b-a807-4030fe58e165-klusterlet-config\") pod \"klusterlet-addon-workmgr-5657cdcbcb-hfmw5\" (UID: \"7d41ac46-970e-446b-a807-4030fe58e165\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:25.002252 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.002217 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k48l9\" (UniqueName: \"kubernetes.io/projected/7d41ac46-970e-446b-a807-4030fe58e165-kube-api-access-k48l9\") pod \"klusterlet-addon-workmgr-5657cdcbcb-hfmw5\" (UID: \"7d41ac46-970e-446b-a807-4030fe58e165\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:25.002999 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.002957 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1467496e-62de-4881-b8a9-9e090e9e0c1b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-wgvx5\" (UID: \"1467496e-62de-4881-b8a9-9e090e9e0c1b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" Jul 22 16:08:25.005843 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.005798 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1467496e-62de-4881-b8a9-9e090e9e0c1b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-wgvx5\" (UID: \"1467496e-62de-4881-b8a9-9e090e9e0c1b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" Jul 22 16:08:25.055284 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.053343 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77fb4c4bf8-xdzrw"] Jul 22 16:08:25.059171 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.059139 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.074515 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.074487 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77fb4c4bf8-xdzrw"] Jul 22 16:08:25.103431 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103318 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7db24c11-eba3-43d4-8491-7f8897046261-registry-certificates\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.103431 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103371 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k48l9\" (UniqueName: \"kubernetes.io/projected/7d41ac46-970e-446b-a807-4030fe58e165-kube-api-access-k48l9\") pod \"klusterlet-addon-workmgr-5657cdcbcb-hfmw5\" (UID: \"7d41ac46-970e-446b-a807-4030fe58e165\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:25.103431 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103410 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7db24c11-eba3-43d4-8491-7f8897046261-trusted-ca\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.103431 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7db24c11-eba3-43d4-8491-7f8897046261-image-registry-private-configuration\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.103789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103465 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d94a6f3-2731-4fc1-9d03-fb98f2f37092-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc\" (UID: \"7d94a6f3-2731-4fc1-9d03-fb98f2f37092\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" Jul 22 16:08:25.103789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103488 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d41ac46-970e-446b-a807-4030fe58e165-tmp\") pod \"klusterlet-addon-workmgr-5657cdcbcb-hfmw5\" (UID: \"7d41ac46-970e-446b-a807-4030fe58e165\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:25.103789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103512 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hz9p\" (UniqueName: \"kubernetes.io/projected/7db24c11-eba3-43d4-8491-7f8897046261-kube-api-access-4hz9p\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.103789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103539 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7db24c11-eba3-43d4-8491-7f8897046261-bound-sa-token\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.103789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103567 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7db24c11-eba3-43d4-8491-7f8897046261-registry-tls\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.103789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103591 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7db24c11-eba3-43d4-8491-7f8897046261-ca-trust-extracted\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.103789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103644 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22745\" (UniqueName: \"kubernetes.io/projected/7d94a6f3-2731-4fc1-9d03-fb98f2f37092-kube-api-access-22745\") pod \"managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc\" (UID: \"7d94a6f3-2731-4fc1-9d03-fb98f2f37092\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" Jul 22 16:08:25.103789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103671 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7d41ac46-970e-446b-a807-4030fe58e165-klusterlet-config\") pod \"klusterlet-addon-workmgr-5657cdcbcb-hfmw5\" (UID: \"7d41ac46-970e-446b-a807-4030fe58e165\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:25.103789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.103704 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7db24c11-eba3-43d4-8491-7f8897046261-installation-pull-secrets\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.104637 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.104556 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7d41ac46-970e-446b-a807-4030fe58e165-tmp\") pod \"klusterlet-addon-workmgr-5657cdcbcb-hfmw5\" (UID: \"7d41ac46-970e-446b-a807-4030fe58e165\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:25.107739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.107694 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d94a6f3-2731-4fc1-9d03-fb98f2f37092-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc\" (UID: \"7d94a6f3-2731-4fc1-9d03-fb98f2f37092\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" Jul 22 16:08:25.108289 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.108244 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7d41ac46-970e-446b-a807-4030fe58e165-klusterlet-config\") pod \"klusterlet-addon-workmgr-5657cdcbcb-hfmw5\" (UID: \"7d41ac46-970e-446b-a807-4030fe58e165\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:25.118386 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.118314 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k48l9\" (UniqueName: \"kubernetes.io/projected/7d41ac46-970e-446b-a807-4030fe58e165-kube-api-access-k48l9\") pod \"klusterlet-addon-workmgr-5657cdcbcb-hfmw5\" (UID: \"7d41ac46-970e-446b-a807-4030fe58e165\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:25.119096 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.119071 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22745\" (UniqueName: \"kubernetes.io/projected/7d94a6f3-2731-4fc1-9d03-fb98f2f37092-kube-api-access-22745\") pod \"managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc\" (UID: \"7d94a6f3-2731-4fc1-9d03-fb98f2f37092\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" Jul 22 16:08:25.133642 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.133601 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wdbjp\"" Jul 22 16:08:25.140072 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.140047 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nhdrz" Jul 22 16:08:25.185961 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.185926 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" Jul 22 16:08:25.204421 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.204367 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7db24c11-eba3-43d4-8491-7f8897046261-installation-pull-secrets\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.204421 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.204420 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7db24c11-eba3-43d4-8491-7f8897046261-registry-certificates\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.204697 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.204469 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7db24c11-eba3-43d4-8491-7f8897046261-trusted-ca\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.204697 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.204496 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7db24c11-eba3-43d4-8491-7f8897046261-image-registry-private-configuration\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.204697 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.204532 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hz9p\" (UniqueName: \"kubernetes.io/projected/7db24c11-eba3-43d4-8491-7f8897046261-kube-api-access-4hz9p\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.204697 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.204560 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7db24c11-eba3-43d4-8491-7f8897046261-bound-sa-token\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.204697 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.204587 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7db24c11-eba3-43d4-8491-7f8897046261-registry-tls\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.204697 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.204617 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7db24c11-eba3-43d4-8491-7f8897046261-ca-trust-extracted\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.205264 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.205028 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7db24c11-eba3-43d4-8491-7f8897046261-ca-trust-extracted\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.205701 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.205677 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7db24c11-eba3-43d4-8491-7f8897046261-registry-certificates\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.206013 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.205990 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7db24c11-eba3-43d4-8491-7f8897046261-trusted-ca\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.208341 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.208320 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7db24c11-eba3-43d4-8491-7f8897046261-registry-tls\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.208981 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.208905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7db24c11-eba3-43d4-8491-7f8897046261-installation-pull-secrets\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.210293 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.210269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7db24c11-eba3-43d4-8491-7f8897046261-image-registry-private-configuration\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.220003 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.219951 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7db24c11-eba3-43d4-8491-7f8897046261-bound-sa-token\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.225791 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.225767 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:25.232848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.232795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hz9p\" (UniqueName: \"kubernetes.io/projected/7db24c11-eba3-43d4-8491-7f8897046261-kube-api-access-4hz9p\") pod \"image-registry-77fb4c4bf8-xdzrw\" (UID: \"7db24c11-eba3-43d4-8491-7f8897046261\") " pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:25.239235 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.239051 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" Jul 22 16:08:25.372043 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:25.371699 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:29.392011 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.391971 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5"] Jul 22 16:08:29.395935 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.395816 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc"] Jul 22 16:08:29.401936 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:29.401303 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d94a6f3_2731_4fc1_9d03_fb98f2f37092.slice/crio-79dae054118b036b53ed93a288d8e8ea98ed0737b14c8510e633ef06ec6975bb WatchSource:0}: Error finding container 79dae054118b036b53ed93a288d8e8ea98ed0737b14c8510e633ef06ec6975bb: Status 404 returned error can't find the container with id 79dae054118b036b53ed93a288d8e8ea98ed0737b14c8510e633ef06ec6975bb Jul 22 16:08:29.411355 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.411326 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nhdrz"] Jul 22 16:08:29.415828 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:29.415798 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7282bf8f_d9c6_4f64_8e4f_ab36bef8369d.slice/crio-deae38dbe26da51bb51b98eb91fd888662c827340c89722821d5dd9e8fff4a9b WatchSource:0}: Error finding container deae38dbe26da51bb51b98eb91fd888662c827340c89722821d5dd9e8fff4a9b: Status 404 returned error can't find the container with id deae38dbe26da51bb51b98eb91fd888662c827340c89722821d5dd9e8fff4a9b Jul 22 16:08:29.441937 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.441900 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j4g57" event={"ID":"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7","Type":"ContainerStarted","Data":"6914f5e3c2125ab65cce9787e402c7faea3f5473959e512c52e6caad072b3b77"} Jul 22 16:08:29.443732 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.443703 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lfc4g" event={"ID":"02186816-0aad-4a17-943b-e09d6dca7a03","Type":"ContainerStarted","Data":"d3bc6f678a4f59db627fed0ff51685d8a2481ec054cd52e1aa43c050faf3319d"} Jul 22 16:08:29.445020 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.444999 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nhdrz" event={"ID":"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d","Type":"ContainerStarted","Data":"deae38dbe26da51bb51b98eb91fd888662c827340c89722821d5dd9e8fff4a9b"} Jul 22 16:08:29.446110 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.446088 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" event={"ID":"7d94a6f3-2731-4fc1-9d03-fb98f2f37092","Type":"ContainerStarted","Data":"79dae054118b036b53ed93a288d8e8ea98ed0737b14c8510e633ef06ec6975bb"} Jul 22 16:08:29.447531 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.447510 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5sllq" event={"ID":"3e191455-0b73-49ef-821c-1c6e74d88494","Type":"ContainerStarted","Data":"01083044c936b956e319436cc711a363c74d842a0328a480eb197db11286db30"} Jul 22 16:08:29.447685 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.447664 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:08:29.448932 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.448914 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7zwc4" event={"ID":"7ce617c4-ea42-4943-8dde-54b9bfc24d0d","Type":"ContainerStarted","Data":"80c04d4def8cbabdc8fa9edb00ed53ff66dd4ba1b18a747b82854f6d1463af8d"} Jul 22 16:08:29.450155 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.450137 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" event={"ID":"1467496e-62de-4881-b8a9-9e090e9e0c1b","Type":"ContainerStarted","Data":"84135c856e143a4fea8347d67936b2782d86b46ebc082922cac355d405c16a05"} Jul 22 16:08:29.464208 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.464160 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bb85c9569-5kqjz"] Jul 22 16:08:29.466021 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:29.465996 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2d0bc0_a54e_4508_9bd3_08facb2435d5.slice/crio-fee1ceae559d32f3851662cc862eee02b38ad4b41a03a009a84bf49675b82991 WatchSource:0}: Error finding container fee1ceae559d32f3851662cc862eee02b38ad4b41a03a009a84bf49675b82991: Status 404 returned error can't find the container with id fee1ceae559d32f3851662cc862eee02b38ad4b41a03a009a84bf49675b82991 Jul 22 16:08:29.478959 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.478909 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lfc4g" podStartSLOduration=71.528406441 podStartE2EDuration="1m12.478893797s" totalImagesPullingTime="950.487356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:29.478756796 +0000 UTC m=+72.874298630" watchObservedRunningTime="2026-07-22 16:08:29.478893797 +0000 UTC m=+72.874435677" Jul 22 16:08:29.510943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.510897 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7zwc4" podStartSLOduration=66.47920204 podStartE2EDuration="1m12.510881s" totalImagesPullingTime="6.03167896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:29.510346749 +0000 UTC m=+72.905888582" watchObservedRunningTime="2026-07-22 16:08:29.510881 +0000 UTC m=+72.906422832" Jul 22 16:08:29.619504 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.618897 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5sllq" podStartSLOduration=66.900118393 podStartE2EDuration="1m12.618876023s" totalImagesPullingTime="5.71875763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:29.536006244 +0000 UTC m=+72.931548081" watchObservedRunningTime="2026-07-22 16:08:29.618876023 +0000 UTC m=+73.014417856" Jul 22 16:08:29.619504 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.619342 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5"] Jul 22 16:08:29.634967 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:29.634936 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77fb4c4bf8-xdzrw"] Jul 22 16:08:29.639173 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:29.639137 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db24c11_eba3_43d4_8491_7f8897046261.slice/crio-6a2aa4023007df582b42b124f0ba005e929d452f30fabeeed4dfef1678c203dc WatchSource:0}: Error finding container 6a2aa4023007df582b42b124f0ba005e929d452f30fabeeed4dfef1678c203dc: Status 404 returned error can't find the container with id 6a2aa4023007df582b42b124f0ba005e929d452f30fabeeed4dfef1678c203dc Jul 22 16:08:30.460612 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:30.460519 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" event={"ID":"7d41ac46-970e-446b-a807-4030fe58e165","Type":"ContainerStarted","Data":"911cc193e4529dce99362138f89f7964055dab12c4f62c5da51f0cc48bf177e5"} Jul 22 16:08:30.466852 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:30.466053 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" event={"ID":"be2d0bc0-a54e-4508-9bd3-08facb2435d5","Type":"ContainerStarted","Data":"cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523"} Jul 22 16:08:30.466852 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:30.466111 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" event={"ID":"be2d0bc0-a54e-4508-9bd3-08facb2435d5","Type":"ContainerStarted","Data":"fee1ceae559d32f3851662cc862eee02b38ad4b41a03a009a84bf49675b82991"} Jul 22 16:08:30.466852 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:30.466372 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:08:30.470852 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:30.470798 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" event={"ID":"7db24c11-eba3-43d4-8491-7f8897046261","Type":"ContainerStarted","Data":"e58ad264c8b5599969d56000a26236f418279c2bdfc8ed0788fc3428a1c9a11c"} Jul 22 16:08:30.470852 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:30.470844 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" event={"ID":"7db24c11-eba3-43d4-8491-7f8897046261","Type":"ContainerStarted","Data":"6a2aa4023007df582b42b124f0ba005e929d452f30fabeeed4dfef1678c203dc"} Jul 22 16:08:30.471302 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:30.471282 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:30.517956 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:30.517896 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" podStartSLOduration=6.517874008 podStartE2EDuration="6.517874008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:30.517352518 +0000 UTC m=+73.912894352" watchObservedRunningTime="2026-07-22 16:08:30.517874008 +0000 UTC m=+73.913415823" Jul 22 16:08:30.518812 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:30.518767 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" podStartSLOduration=46.518753069 podStartE2EDuration="46.518753069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:30.49183105 +0000 UTC m=+73.887372883" watchObservedRunningTime="2026-07-22 16:08:30.518753069 +0000 UTC m=+73.914294903" Jul 22 16:08:31.702347 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:31.701889 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm"] Jul 22 16:08:31.705587 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:31.704181 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" Jul 22 16:08:31.708762 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:31.708117 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 22 16:08:31.708762 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:31.708394 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cjg6n\"" Jul 22 16:08:31.721444 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:31.721366 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm"] Jul 22 16:08:31.764746 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:31.764700 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8636682-8baf-4b64-bd7f-29d7de537cfe-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-rkdpm\" (UID: \"a8636682-8baf-4b64-bd7f-29d7de537cfe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" Jul 22 16:08:31.866346 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:31.865971 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8636682-8baf-4b64-bd7f-29d7de537cfe-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-rkdpm\" (UID: \"a8636682-8baf-4b64-bd7f-29d7de537cfe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" Jul 22 16:08:31.866346 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:31.866093 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 22 16:08:31.866346 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:31.866145 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8636682-8baf-4b64-bd7f-29d7de537cfe-tls-certificates podName:a8636682-8baf-4b64-bd7f-29d7de537cfe nodeName:}" failed. No retries permitted until 2026-07-22 16:08:32.36612914 +0000 UTC m=+75.761670952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a8636682-8baf-4b64-bd7f-29d7de537cfe-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-rkdpm" (UID: "a8636682-8baf-4b64-bd7f-29d7de537cfe") : secret "prometheus-operator-admission-webhook-tls" not found Jul 22 16:08:32.371265 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:32.371227 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8636682-8baf-4b64-bd7f-29d7de537cfe-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-rkdpm\" (UID: \"a8636682-8baf-4b64-bd7f-29d7de537cfe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" Jul 22 16:08:32.374017 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:32.373982 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a8636682-8baf-4b64-bd7f-29d7de537cfe-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-rkdpm\" (UID: \"a8636682-8baf-4b64-bd7f-29d7de537cfe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" Jul 22 16:08:32.618127 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:32.618089 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" Jul 22 16:08:36.062901 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.062866 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm"] Jul 22 16:08:36.067840 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:36.067804 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8636682_8baf_4b64_bd7f_29d7de537cfe.slice/crio-c239a5f01c0a1e07059038c59204189295905737b61fe73b125fa65379677f5d WatchSource:0}: Error finding container c239a5f01c0a1e07059038c59204189295905737b61fe73b125fa65379677f5d: Status 404 returned error can't find the container with id c239a5f01c0a1e07059038c59204189295905737b61fe73b125fa65379677f5d Jul 22 16:08:36.499754 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.499714 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" event={"ID":"1467496e-62de-4881-b8a9-9e090e9e0c1b","Type":"ContainerStarted","Data":"b90345b0c90e1caf7690f47134b9afc4914087169703b8573b821ae895d3a6ea"} Jul 22 16:08:36.501860 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.501830 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j4g57" event={"ID":"201d6928-9fae-43b8-b0a4-d1ce64c9d9c7","Type":"ContainerStarted","Data":"6b49e021eba37fa112987c76ae146eae1236a6d448f7e58b9488a95c85a26c25"} Jul 22 16:08:36.503680 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.503652 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nhdrz" event={"ID":"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d","Type":"ContainerStarted","Data":"3a54eaee0abf3ac73d3594356da4ca39aa3bc7995ff675c6dde8985487a0a833"} Jul 22 16:08:36.503790 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.503708 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nhdrz" Jul 22 16:08:36.503790 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.503731 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nhdrz" event={"ID":"7282bf8f-d9c6-4f64-8e4f-ab36bef8369d","Type":"ContainerStarted","Data":"8e6a02b64cf2901ca14de56044a4f1533941fe3c36401146060649bba2c662b7"} Jul 22 16:08:36.505002 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.504980 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" event={"ID":"7d94a6f3-2731-4fc1-9d03-fb98f2f37092","Type":"ContainerStarted","Data":"6052f4818a6cd1654fdabc070bb3c410a73aa18c7b505f4f16f299239924b61c"} Jul 22 16:08:36.506330 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.506309 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" event={"ID":"7d41ac46-970e-446b-a807-4030fe58e165","Type":"ContainerStarted","Data":"558e302f94355af9a7f9d1a752b262cb3d8ce11e8c00b7e76a3bb3b2218d04b2"} Jul 22 16:08:36.506516 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.506497 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:36.507475 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.507453 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" event={"ID":"a8636682-8baf-4b64-bd7f-29d7de537cfe","Type":"ContainerStarted","Data":"c239a5f01c0a1e07059038c59204189295905737b61fe73b125fa65379677f5d"} Jul 22 16:08:36.508243 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.508226 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" Jul 22 16:08:36.524609 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.524568 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-wgvx5" podStartSLOduration=6.04478614 podStartE2EDuration="12.524556161s" totalImagesPullingTime="6.479770021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:36.523398248 +0000 UTC m=+79.918940084" watchObservedRunningTime="2026-07-22 16:08:36.524556161 +0000 UTC m=+79.920097990" Jul 22 16:08:36.566429 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.566380 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5657cdcbcb-hfmw5" podStartSLOduration=6.282122064 podStartE2EDuration="12.566363919s" totalImagesPullingTime="6.284241855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:36.565712673 +0000 UTC m=+79.961254506" watchObservedRunningTime="2026-07-22 16:08:36.566363919 +0000 UTC m=+79.961905730" Jul 22 16:08:36.566693 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.566670 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-67bb8f84f4-4dvtc" podStartSLOduration=6.072881064 podStartE2EDuration="12.566662095s" totalImagesPullingTime="6.493781031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:36.544896618 +0000 UTC m=+79.940438450" watchObservedRunningTime="2026-07-22 16:08:36.566662095 +0000 UTC m=+79.962203927" Jul 22 16:08:36.590448 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.590399 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nhdrz" podStartSLOduration=38.122804131 podStartE2EDuration="44.590383189s" totalImagesPullingTime="6.467579058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:36.589436793 +0000 UTC m=+79.984978625" watchObservedRunningTime="2026-07-22 16:08:36.590383189 +0000 UTC m=+79.985925020" Jul 22 16:08:36.619029 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:36.618975 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-j4g57" podStartSLOduration=17.197871104 podStartE2EDuration="35.618957922s" totalImagesPullingTime="18.421086818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:08:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:36.617504032 +0000 UTC m=+80.013045891" watchObservedRunningTime="2026-07-22 16:08:36.618957922 +0000 UTC m=+80.014499754" Jul 22 16:08:37.512750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:37.512691 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" event={"ID":"a8636682-8baf-4b64-bd7f-29d7de537cfe","Type":"ContainerStarted","Data":"7af93f7d903aad4be283a8529f1035096ecedef81f3361a11fe0e8cc3b3bb785"} Jul 22 16:08:37.513610 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:37.513592 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" Jul 22 16:08:37.517513 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:37.517492 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" Jul 22 16:08:37.530236 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:37.530189 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-rkdpm" podStartSLOduration=5.390831545 podStartE2EDuration="6.530174791s" totalImagesPullingTime="1.139343246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:37.529151031 +0000 UTC m=+80.924692874" watchObservedRunningTime="2026-07-22 16:08:37.530174791 +0000 UTC m=+80.925716623" Jul 22 16:08:42.159284 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.159168 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dwv8m"] Jul 22 16:08:42.162149 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.162128 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.164938 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.164913 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 16:08:42.164938 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.164938 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 16:08:42.165103 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.164939 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 16:08:42.165103 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.165000 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jqfxs\"" Jul 22 16:08:42.165103 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.164939 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 16:08:42.165103 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.164920 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 16:08:42.165766 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.165749 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 16:08:42.250820 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.250783 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-accelerators-collector-config\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.250820 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.250828 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-textfile\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.251075 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.250849 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48066dc6-1e34-48dd-bfcf-9334dbbe2943-sys\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.251075 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.250920 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-tls\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.251075 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.250958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48066dc6-1e34-48dd-bfcf-9334dbbe2943-metrics-client-ca\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.251075 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.250994 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/48066dc6-1e34-48dd-bfcf-9334dbbe2943-root\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.251075 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.251053 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-wtmp\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.251214 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.251098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.251214 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.251115 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwp7z\" (UniqueName: \"kubernetes.io/projected/48066dc6-1e34-48dd-bfcf-9334dbbe2943-kube-api-access-jwp7z\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351692 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351652 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-textfile\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351692 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48066dc6-1e34-48dd-bfcf-9334dbbe2943-sys\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351722 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-tls\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351742 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48066dc6-1e34-48dd-bfcf-9334dbbe2943-metrics-client-ca\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351767 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/48066dc6-1e34-48dd-bfcf-9334dbbe2943-root\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351776 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48066dc6-1e34-48dd-bfcf-9334dbbe2943-sys\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351792 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-wtmp\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwp7z\" (UniqueName: \"kubernetes.io/projected/48066dc6-1e34-48dd-bfcf-9334dbbe2943-kube-api-access-jwp7z\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351895 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-wtmp\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.351939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351894 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/48066dc6-1e34-48dd-bfcf-9334dbbe2943-root\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.352265 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.351993 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-accelerators-collector-config\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.352265 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.352079 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-textfile\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.352376 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.352357 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48066dc6-1e34-48dd-bfcf-9334dbbe2943-metrics-client-ca\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.352460 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.352444 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-accelerators-collector-config\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.354144 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.354125 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.354250 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.354235 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/48066dc6-1e34-48dd-bfcf-9334dbbe2943-node-exporter-tls\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.374923 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.374885 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwp7z\" (UniqueName: \"kubernetes.io/projected/48066dc6-1e34-48dd-bfcf-9334dbbe2943-kube-api-access-jwp7z\") pod \"node-exporter-dwv8m\" (UID: \"48066dc6-1e34-48dd-bfcf-9334dbbe2943\") " pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.472743 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.472714 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dwv8m" Jul 22 16:08:42.482119 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:42.482089 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48066dc6_1e34_48dd_bfcf_9334dbbe2943.slice/crio-10e6e127cd71da388078fa7473ecafe48d7935020b11a325fa8c3d15b4cfe470 WatchSource:0}: Error finding container 10e6e127cd71da388078fa7473ecafe48d7935020b11a325fa8c3d15b4cfe470: Status 404 returned error can't find the container with id 10e6e127cd71da388078fa7473ecafe48d7935020b11a325fa8c3d15b4cfe470 Jul 22 16:08:42.530956 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:42.530918 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwv8m" event={"ID":"48066dc6-1e34-48dd-bfcf-9334dbbe2943","Type":"ContainerStarted","Data":"10e6e127cd71da388078fa7473ecafe48d7935020b11a325fa8c3d15b4cfe470"} Jul 22 16:08:43.536112 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:43.536077 2564 generic.go:356] "Generic (PLEG): container finished" podID="48066dc6-1e34-48dd-bfcf-9334dbbe2943" containerID="40230feef7d6e73e3bbf14a6229da91e325fee52e89a425ec98047e2e320d5ac" exitCode=0 Jul 22 16:08:43.536496 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:43.536141 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwv8m" event={"ID":"48066dc6-1e34-48dd-bfcf-9334dbbe2943","Type":"ContainerDied","Data":"40230feef7d6e73e3bbf14a6229da91e325fee52e89a425ec98047e2e320d5ac"} Jul 22 16:08:44.541998 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:44.541956 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwv8m" event={"ID":"48066dc6-1e34-48dd-bfcf-9334dbbe2943","Type":"ContainerStarted","Data":"4efd7a1b3274f85fc04b7131f483b98d6ba7b394b3fc6bbea3a2593cad61ffb3"} Jul 22 16:08:44.542372 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:44.542011 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dwv8m" event={"ID":"48066dc6-1e34-48dd-bfcf-9334dbbe2943","Type":"ContainerStarted","Data":"43dfd8950bc1c2d3345559bd2d3dbea824acc0b2ab7611803d397aa45def452e"} Jul 22 16:08:44.566781 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:44.566731 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dwv8m" podStartSLOduration=1.651551745 podStartE2EDuration="2.566716s" totalImagesPullingTime="850.300796ms" totalInitContainerRuntime="64.863459ms" isStatefulPod=true podCreationTimestamp="2026-07-22 16:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:44.564888131 +0000 UTC m=+87.960429963" watchObservedRunningTime="2026-07-22 16:08:44.566716 +0000 UTC m=+87.962257832" Jul 22 16:08:45.219172 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.219133 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7"] Jul 22 16:08:45.221598 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.221574 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.224228 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.224208 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 22 16:08:45.224355 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.224209 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 22 16:08:45.224355 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.224209 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 22 16:08:45.224698 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.224685 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-wjj47\"" Jul 22 16:08:45.224796 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.224777 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 22 16:08:45.225240 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.225225 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ccbpr9bbajffn\"" Jul 22 16:08:45.225275 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.225249 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 22 16:08:45.239451 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.239420 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7"] Jul 22 16:08:45.376002 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.375961 2564 patch_prober.go:28] interesting pod/image-registry-77fb4c4bf8-xdzrw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 22 16:08:45.376174 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.376022 2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" podUID="7db24c11-eba3-43d4-8491-7f8897046261" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 16:08:45.376371 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.376348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-grpc-tls\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.376469 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.376451 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.376525 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.376497 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.376570 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.376536 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-tls\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.376609 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.376590 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tszdv\" (UniqueName: \"kubernetes.io/projected/f991ce8c-0f02-4f1a-8209-a548098c3126-kube-api-access-tszdv\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.376664 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.376617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f991ce8c-0f02-4f1a-8209-a548098c3126-metrics-client-ca\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.376664 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.376658 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.376726 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.376679 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.477930 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.477895 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f991ce8c-0f02-4f1a-8209-a548098c3126-metrics-client-ca\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.478128 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.477937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.478128 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.477966 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.478128 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.478016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-grpc-tls\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.478128 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.478065 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.478128 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.478099 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.478365 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.478133 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-tls\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.478365 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.478163 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tszdv\" (UniqueName: \"kubernetes.io/projected/f991ce8c-0f02-4f1a-8209-a548098c3126-kube-api-access-tszdv\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.478804 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.478775 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f991ce8c-0f02-4f1a-8209-a548098c3126-metrics-client-ca\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.480963 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.480916 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.481306 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.481275 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.481408 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.481287 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.481408 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.481336 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-grpc-tls\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.481408 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.481354 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-tls\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.481509 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.481424 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f991ce8c-0f02-4f1a-8209-a548098c3126-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.497518 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.497490 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tszdv\" (UniqueName: \"kubernetes.io/projected/f991ce8c-0f02-4f1a-8209-a548098c3126-kube-api-access-tszdv\") pod \"thanos-querier-5b558fdcbf-7r8l7\" (UID: \"f991ce8c-0f02-4f1a-8209-a548098c3126\") " pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.531216 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.531179 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:45.671333 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:45.671310 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7"] Jul 22 16:08:45.673114 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:45.673088 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf991ce8c_0f02_4f1a_8209_a548098c3126.slice/crio-17dfc5b3ae8ceeed4086237fc481c583dc3a2f01ed8fa9e62819dab7093fd391 WatchSource:0}: Error finding container 17dfc5b3ae8ceeed4086237fc481c583dc3a2f01ed8fa9e62819dab7093fd391: Status 404 returned error can't find the container with id 17dfc5b3ae8ceeed4086237fc481c583dc3a2f01ed8fa9e62819dab7093fd391 Jul 22 16:08:46.515386 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:46.515356 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nhdrz" Jul 22 16:08:46.550981 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:46.550945 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" event={"ID":"f991ce8c-0f02-4f1a-8209-a548098c3126","Type":"ContainerStarted","Data":"17dfc5b3ae8ceeed4086237fc481c583dc3a2f01ed8fa9e62819dab7093fd391"} Jul 22 16:08:47.713476 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:47.713450 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-rqbx8"] Jul 22 16:08:47.715270 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:47.715255 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-rqbx8" Jul 22 16:08:47.719708 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:47.719663 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 16:08:47.719826 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:47.719668 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9l24m\"" Jul 22 16:08:47.719826 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:47.719666 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 16:08:47.733893 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:47.733865 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-rqbx8"] Jul 22 16:08:47.798573 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:47.798537 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbh5t\" (UniqueName: \"kubernetes.io/projected/aa4d869d-91fb-45e6-9466-f0bbd0ae957c-kube-api-access-bbh5t\") pod \"downloads-5bc67dcd46-rqbx8\" (UID: \"aa4d869d-91fb-45e6-9466-f0bbd0ae957c\") " pod="openshift-console/downloads-5bc67dcd46-rqbx8" Jul 22 16:08:47.899008 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:47.898967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbh5t\" (UniqueName: \"kubernetes.io/projected/aa4d869d-91fb-45e6-9466-f0bbd0ae957c-kube-api-access-bbh5t\") pod \"downloads-5bc67dcd46-rqbx8\" (UID: \"aa4d869d-91fb-45e6-9466-f0bbd0ae957c\") " pod="openshift-console/downloads-5bc67dcd46-rqbx8" Jul 22 16:08:47.923248 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:47.923180 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbh5t\" (UniqueName: \"kubernetes.io/projected/aa4d869d-91fb-45e6-9466-f0bbd0ae957c-kube-api-access-bbh5t\") pod \"downloads-5bc67dcd46-rqbx8\" (UID: \"aa4d869d-91fb-45e6-9466-f0bbd0ae957c\") " pod="openshift-console/downloads-5bc67dcd46-rqbx8" Jul 22 16:08:48.034463 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:48.034429 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-rqbx8" Jul 22 16:08:48.165805 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:48.165776 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-rqbx8"] Jul 22 16:08:48.168047 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:48.168018 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4d869d_91fb_45e6_9466_f0bbd0ae957c.slice/crio-bc22620de2e4cbda4c31339ab40da456ce0e74697b4adf610cb2505b8d5eb7ea WatchSource:0}: Error finding container bc22620de2e4cbda4c31339ab40da456ce0e74697b4adf610cb2505b8d5eb7ea: Status 404 returned error can't find the container with id bc22620de2e4cbda4c31339ab40da456ce0e74697b4adf610cb2505b8d5eb7ea Jul 22 16:08:48.560547 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:48.560510 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" event={"ID":"f991ce8c-0f02-4f1a-8209-a548098c3126","Type":"ContainerStarted","Data":"be8fa3aa5ad4999144270d03b3d7bda1c21aee8f4061413113af1efe270636c0"} Jul 22 16:08:48.560730 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:48.560584 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" event={"ID":"f991ce8c-0f02-4f1a-8209-a548098c3126","Type":"ContainerStarted","Data":"9db6c792f224206b4a535bd60558382c51ee729dafd42fb8b679126dce405acd"} Jul 22 16:08:48.560730 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:48.560636 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" event={"ID":"f991ce8c-0f02-4f1a-8209-a548098c3126","Type":"ContainerStarted","Data":"30a1d93732acfa0ccc3c3b8321599f4cbbc1a3d3130307772b4c281dd0850690"} Jul 22 16:08:48.561689 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:48.561657 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-rqbx8" event={"ID":"aa4d869d-91fb-45e6-9466-f0bbd0ae957c","Type":"ContainerStarted","Data":"bc22620de2e4cbda4c31339ab40da456ce0e74697b4adf610cb2505b8d5eb7ea"} Jul 22 16:08:49.571353 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:49.571320 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" event={"ID":"f991ce8c-0f02-4f1a-8209-a548098c3126","Type":"ContainerStarted","Data":"240e1c953ae872cc77b43e755677e39295f2dec69a46d5e46e022b2d539aba01"} Jul 22 16:08:49.571845 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:49.571394 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" event={"ID":"f991ce8c-0f02-4f1a-8209-a548098c3126","Type":"ContainerStarted","Data":"a0502e8fdaa549c70154327bd5c06502dd52561524a2f0726a906a59bf59afa1"} Jul 22 16:08:49.571845 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:49.571425 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" event={"ID":"f991ce8c-0f02-4f1a-8209-a548098c3126","Type":"ContainerStarted","Data":"31dc7bd9d3efa12c45984fc4f54e29505e7f3818f0ed85483227efed461989ea"} Jul 22 16:08:49.571845 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:49.571504 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:49.601564 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:49.601493 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" podStartSLOduration=1.518467766 podStartE2EDuration="4.601470706s" totalImagesPullingTime="3.08300294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:08:49.599406035 +0000 UTC m=+92.994947880" watchObservedRunningTime="2026-07-22 16:08:49.601470706 +0000 UTC m=+92.997012540" Jul 22 16:08:50.472593 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:50.472561 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:08:52.483697 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:52.483671 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77fb4c4bf8-xdzrw" Jul 22 16:08:55.507422 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.507356 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" podUID="be2d0bc0-a54e-4508-9bd3-08facb2435d5" containerName="registry" containerID="cri-o://cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523" gracePeriod=30 Jul 22 16:08:55.581835 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.581772 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b558fdcbf-7r8l7" Jul 22 16:08:55.800641 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.800598 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:08:55.968904 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.968865 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-bound-sa-token\") pod \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " Jul 22 16:08:55.969100 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.968923 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") pod \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " Jul 22 16:08:55.969100 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.968958 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be2d0bc0-a54e-4508-9bd3-08facb2435d5-ca-trust-extracted\") pod \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " Jul 22 16:08:55.969100 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.968985 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4txh\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-kube-api-access-s4txh\") pod \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " Jul 22 16:08:55.969100 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.969022 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-image-registry-private-configuration\") pod \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " Jul 22 16:08:55.969100 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.969085 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-installation-pull-secrets\") pod \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " Jul 22 16:08:55.969345 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.969126 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-trusted-ca\") pod \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " Jul 22 16:08:55.969345 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.969151 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-certificates\") pod \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\" (UID: \"be2d0bc0-a54e-4508-9bd3-08facb2435d5\") " Jul 22 16:08:55.969760 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.969596 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "be2d0bc0-a54e-4508-9bd3-08facb2435d5" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:08:55.969760 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.969727 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "be2d0bc0-a54e-4508-9bd3-08facb2435d5" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:08:55.971639 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.971575 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "be2d0bc0-a54e-4508-9bd3-08facb2435d5" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:08:55.971727 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.971682 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "be2d0bc0-a54e-4508-9bd3-08facb2435d5" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:08:55.971844 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.971821 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "be2d0bc0-a54e-4508-9bd3-08facb2435d5" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:08:55.971979 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.971960 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-kube-api-access-s4txh" (OuterVolumeSpecName: "kube-api-access-s4txh") pod "be2d0bc0-a54e-4508-9bd3-08facb2435d5" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5"). InnerVolumeSpecName "kube-api-access-s4txh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:08:55.972017 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.971968 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "be2d0bc0-a54e-4508-9bd3-08facb2435d5" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:08:55.978238 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:55.978205 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be2d0bc0-a54e-4508-9bd3-08facb2435d5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "be2d0bc0-a54e-4508-9bd3-08facb2435d5" (UID: "be2d0bc0-a54e-4508-9bd3-08facb2435d5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:08:56.070376 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.070292 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-installation-pull-secrets\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:08:56.070376 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.070327 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-trusted-ca\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:08:56.070376 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.070342 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-certificates\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:08:56.070376 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.070355 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-bound-sa-token\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:08:56.070376 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.070369 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-registry-tls\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:08:56.070376 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.070382 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be2d0bc0-a54e-4508-9bd3-08facb2435d5-ca-trust-extracted\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:08:56.070769 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.070395 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4txh\" (UniqueName: \"kubernetes.io/projected/be2d0bc0-a54e-4508-9bd3-08facb2435d5-kube-api-access-s4txh\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:08:56.070769 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.070410 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be2d0bc0-a54e-4508-9bd3-08facb2435d5-image-registry-private-configuration\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:08:56.600798 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.600698 2564 generic.go:356] "Generic (PLEG): container finished" podID="be2d0bc0-a54e-4508-9bd3-08facb2435d5" containerID="cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523" exitCode=0 Jul 22 16:08:56.600798 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.600753 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" Jul 22 16:08:56.600798 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.600789 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" event={"ID":"be2d0bc0-a54e-4508-9bd3-08facb2435d5","Type":"ContainerDied","Data":"cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523"} Jul 22 16:08:56.601286 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.600846 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bb85c9569-5kqjz" event={"ID":"be2d0bc0-a54e-4508-9bd3-08facb2435d5","Type":"ContainerDied","Data":"fee1ceae559d32f3851662cc862eee02b38ad4b41a03a009a84bf49675b82991"} Jul 22 16:08:56.601286 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.600872 2564 scope.go:117] "RemoveContainer" containerID="cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523" Jul 22 16:08:56.611016 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.610979 2564 scope.go:117] "RemoveContainer" containerID="cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523" Jul 22 16:08:56.611376 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:08:56.611347 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523\": container with ID starting with cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523 not found: ID does not exist" containerID="cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523" Jul 22 16:08:56.611478 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.611383 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523"} err="failed to get container status \"cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523\": rpc error: code = NotFound desc = could not find container \"cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523\": container with ID starting with cd422df9ca21e33e7c3efed4aed754ab04b7a87ed5c3c01d25084f3571d89523 not found: ID does not exist" Jul 22 16:08:56.629831 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.629796 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bb85c9569-5kqjz"] Jul 22 16:08:56.637179 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.637141 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6bb85c9569-5kqjz"] Jul 22 16:08:56.845005 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.844970 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64f7ff79bd-p7q4x"] Jul 22 16:08:56.845280 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.845261 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2d0bc0-a54e-4508-9bd3-08facb2435d5" containerName="registry" Jul 22 16:08:56.845280 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.845277 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2d0bc0-a54e-4508-9bd3-08facb2435d5" containerName="registry" Jul 22 16:08:56.845448 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.845336 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="be2d0bc0-a54e-4508-9bd3-08facb2435d5" containerName="registry" Jul 22 16:08:56.889448 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.889363 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f7ff79bd-p7q4x"] Jul 22 16:08:56.889607 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.889477 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:56.892882 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.892855 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 16:08:56.893011 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.892991 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 16:08:56.893066 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.893053 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6x7rm\"" Jul 22 16:08:56.893149 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.893127 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 16:08:56.893608 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.893592 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 16:08:56.893764 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:56.893748 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 16:08:57.080042 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.080004 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-service-ca\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.080042 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.080048 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-config\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.080259 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.080071 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-serving-cert\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.080259 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.080174 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-oauth-config\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.080259 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.080222 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-oauth-serving-cert\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.080394 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.080301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5xbv\" (UniqueName: \"kubernetes.io/projected/60194dbb-1590-47af-b9da-aa94ffbe0d1f-kube-api-access-z5xbv\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.128716 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.128676 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be2d0bc0-a54e-4508-9bd3-08facb2435d5" path="/var/lib/kubelet/pods/be2d0bc0-a54e-4508-9bd3-08facb2435d5/volumes" Jul 22 16:08:57.181270 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.181174 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-oauth-config\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.181270 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.181229 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-oauth-serving-cert\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.181572 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.181280 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5xbv\" (UniqueName: \"kubernetes.io/projected/60194dbb-1590-47af-b9da-aa94ffbe0d1f-kube-api-access-z5xbv\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.181572 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.181318 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-service-ca\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.181572 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.181347 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-config\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.181572 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.181377 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-serving-cert\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.182110 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.182052 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-oauth-serving-cert\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.182234 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.182207 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-service-ca\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.182234 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.182221 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-config\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.183915 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.183893 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-serving-cert\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.184001 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.183959 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-oauth-config\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.191554 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.191532 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5xbv\" (UniqueName: \"kubernetes.io/projected/60194dbb-1590-47af-b9da-aa94ffbe0d1f-kube-api-access-z5xbv\") pod \"console-64f7ff79bd-p7q4x\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.198421 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.198395 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:08:57.379987 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.379834 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f7ff79bd-p7q4x"] Jul 22 16:08:57.382687 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:08:57.382657 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60194dbb_1590_47af_b9da_aa94ffbe0d1f.slice/crio-0ba2465427e89b872153373e570eb7268d47370d306a6cafb7beb1742c3e9df9 WatchSource:0}: Error finding container 0ba2465427e89b872153373e570eb7268d47370d306a6cafb7beb1742c3e9df9: Status 404 returned error can't find the container with id 0ba2465427e89b872153373e570eb7268d47370d306a6cafb7beb1742c3e9df9 Jul 22 16:08:57.606071 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:08:57.606029 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f7ff79bd-p7q4x" event={"ID":"60194dbb-1590-47af-b9da-aa94ffbe0d1f","Type":"ContainerStarted","Data":"0ba2465427e89b872153373e570eb7268d47370d306a6cafb7beb1742c3e9df9"} Jul 22 16:09:00.476034 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:00.476005 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5sllq" Jul 22 16:09:04.405482 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.405443 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5455985cf7-z8x72"] Jul 22 16:09:04.478426 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.478388 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5455985cf7-z8x72"] Jul 22 16:09:04.478615 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.478536 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.498861 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.498832 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 16:09:04.539134 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.539093 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-oauth-serving-cert\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.539134 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.539131 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-serving-cert\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.539362 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.539182 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-oauth-config\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.539362 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.539197 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgs4g\" (UniqueName: \"kubernetes.io/projected/af6f1cc7-7692-43b5-b27e-c0f66584db77-kube-api-access-rgs4g\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.539362 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.539232 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-service-ca\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.539362 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.539266 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-config\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.539362 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.539282 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-trusted-ca-bundle\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.639943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.639909 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-oauth-config\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.639943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.639944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgs4g\" (UniqueName: \"kubernetes.io/projected/af6f1cc7-7692-43b5-b27e-c0f66584db77-kube-api-access-rgs4g\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.640187 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.640003 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-service-ca\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.640187 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.640060 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-config\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.640187 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.640085 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-trusted-ca-bundle\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.640187 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.640137 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-oauth-serving-cert\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.640187 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.640160 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-serving-cert\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.641026 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.640939 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-service-ca\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.641168 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.641039 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-config\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.641168 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.641062 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-oauth-serving-cert\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.641282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.641250 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-trusted-ca-bundle\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.643302 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.643280 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-serving-cert\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.643419 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.643352 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-oauth-config\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.649586 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.649557 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgs4g\" (UniqueName: \"kubernetes.io/projected/af6f1cc7-7692-43b5-b27e-c0f66584db77-kube-api-access-rgs4g\") pod \"console-5455985cf7-z8x72\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:04.790746 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:04.790703 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:07.446647 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.446603 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5455985cf7-z8x72"] Jul 22 16:09:07.449188 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:09:07.449165 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6f1cc7_7692_43b5_b27e_c0f66584db77.slice/crio-cafc5a18668ff482804abcdf25a4119e0a61579172ced5db041549ed1d532364 WatchSource:0}: Error finding container cafc5a18668ff482804abcdf25a4119e0a61579172ced5db041549ed1d532364: Status 404 returned error can't find the container with id cafc5a18668ff482804abcdf25a4119e0a61579172ced5db041549ed1d532364 Jul 22 16:09:07.650076 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.650034 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-rqbx8" event={"ID":"aa4d869d-91fb-45e6-9466-f0bbd0ae957c","Type":"ContainerStarted","Data":"de330e764a112b1299eb92be29870d7d4dda260191b602a2871b1b44de4b8f0f"} Jul 22 16:09:07.650269 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.650213 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-rqbx8" Jul 22 16:09:07.651479 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.651401 2564 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-rqbx8 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.16:8080/\": dial tcp 10.133.0.16:8080: connect: connection refused" start-of-body= Jul 22 16:09:07.651479 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.651451 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-rqbx8" podUID="aa4d869d-91fb-45e6-9466-f0bbd0ae957c" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.16:8080/\": dial tcp 10.133.0.16:8080: connect: connection refused" Jul 22 16:09:07.651817 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.651793 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f7ff79bd-p7q4x" event={"ID":"60194dbb-1590-47af-b9da-aa94ffbe0d1f","Type":"ContainerStarted","Data":"e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6"} Jul 22 16:09:07.653490 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.653466 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5455985cf7-z8x72" event={"ID":"af6f1cc7-7692-43b5-b27e-c0f66584db77","Type":"ContainerStarted","Data":"5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5"} Jul 22 16:09:07.653583 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.653504 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5455985cf7-z8x72" event={"ID":"af6f1cc7-7692-43b5-b27e-c0f66584db77","Type":"ContainerStarted","Data":"cafc5a18668ff482804abcdf25a4119e0a61579172ced5db041549ed1d532364"} Jul 22 16:09:07.667106 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.667040 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-rqbx8" podStartSLOduration=1.511480009 podStartE2EDuration="20.667021655s" totalImagesPullingTime="19.155541646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:09:07.666459036 +0000 UTC m=+111.062000868" watchObservedRunningTime="2026-07-22 16:09:07.667021655 +0000 UTC m=+111.062563490" Jul 22 16:09:07.686340 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.686288 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5455985cf7-z8x72" podStartSLOduration=3.686269263 podStartE2EDuration="3.686269263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:09:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:09:07.685112164 +0000 UTC m=+111.080654007" watchObservedRunningTime="2026-07-22 16:09:07.686269263 +0000 UTC m=+111.081811094" Jul 22 16:09:07.710922 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:07.710856 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64f7ff79bd-p7q4x" podStartSLOduration=1.781929589 podStartE2EDuration="11.710838632s" totalImagesPullingTime="9.928909043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:09:07.70979911 +0000 UTC m=+111.105340965" watchObservedRunningTime="2026-07-22 16:09:07.710838632 +0000 UTC m=+111.106380465" Jul 22 16:09:08.672995 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:08.672964 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-rqbx8" Jul 22 16:09:14.791549 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:14.791507 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:14.792147 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:14.791910 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:14.797479 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:14.797456 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:15.694415 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:15.694382 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:09:15.746320 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:15.746279 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f7ff79bd-p7q4x"] Jul 22 16:09:17.198749 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:17.198702 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:09:40.771247 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:40.771199 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64f7ff79bd-p7q4x" podUID="60194dbb-1590-47af-b9da-aa94ffbe0d1f" containerName="console" containerID="cri-o://e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6" gracePeriod=15 Jul 22 16:09:41.044795 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.044708 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:09:41.166756 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.166710 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-serving-cert\") pod \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " Jul 22 16:09:41.166756 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.166758 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-oauth-config\") pod \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " Jul 22 16:09:41.167005 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.166831 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-config\") pod \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " Jul 22 16:09:41.167005 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.166867 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5xbv\" (UniqueName: \"kubernetes.io/projected/60194dbb-1590-47af-b9da-aa94ffbe0d1f-kube-api-access-z5xbv\") pod \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " Jul 22 16:09:41.167005 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.166909 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-service-ca\") pod \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " Jul 22 16:09:41.167005 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.166932 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-oauth-serving-cert\") pod \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\" (UID: \"60194dbb-1590-47af-b9da-aa94ffbe0d1f\") " Jul 22 16:09:41.167247 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.167212 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-config" (OuterVolumeSpecName: "console-config") pod "60194dbb-1590-47af-b9da-aa94ffbe0d1f" (UID: "60194dbb-1590-47af-b9da-aa94ffbe0d1f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:09:41.167386 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.167342 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-service-ca" (OuterVolumeSpecName: "service-ca") pod "60194dbb-1590-47af-b9da-aa94ffbe0d1f" (UID: "60194dbb-1590-47af-b9da-aa94ffbe0d1f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:09:41.167386 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.167353 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "60194dbb-1590-47af-b9da-aa94ffbe0d1f" (UID: "60194dbb-1590-47af-b9da-aa94ffbe0d1f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:09:41.169186 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.169160 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "60194dbb-1590-47af-b9da-aa94ffbe0d1f" (UID: "60194dbb-1590-47af-b9da-aa94ffbe0d1f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:09:41.169186 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.169175 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "60194dbb-1590-47af-b9da-aa94ffbe0d1f" (UID: "60194dbb-1590-47af-b9da-aa94ffbe0d1f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:09:41.169355 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.169206 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60194dbb-1590-47af-b9da-aa94ffbe0d1f-kube-api-access-z5xbv" (OuterVolumeSpecName: "kube-api-access-z5xbv") pod "60194dbb-1590-47af-b9da-aa94ffbe0d1f" (UID: "60194dbb-1590-47af-b9da-aa94ffbe0d1f"). InnerVolumeSpecName "kube-api-access-z5xbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:09:41.267848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.267807 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-config\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:09:41.267848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.267840 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5xbv\" (UniqueName: \"kubernetes.io/projected/60194dbb-1590-47af-b9da-aa94ffbe0d1f-kube-api-access-z5xbv\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:09:41.267848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.267851 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-service-ca\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:09:41.267848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.267859 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60194dbb-1590-47af-b9da-aa94ffbe0d1f-oauth-serving-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:09:41.268123 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.267871 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-serving-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:09:41.268123 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.267881 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60194dbb-1590-47af-b9da-aa94ffbe0d1f-console-oauth-config\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:09:41.790858 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.790822 2564 generic.go:356] "Generic (PLEG): container finished" podID="60194dbb-1590-47af-b9da-aa94ffbe0d1f" containerID="e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6" exitCode=0 Jul 22 16:09:41.791338 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.790873 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f7ff79bd-p7q4x" Jul 22 16:09:41.791338 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.790897 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f7ff79bd-p7q4x" event={"ID":"60194dbb-1590-47af-b9da-aa94ffbe0d1f","Type":"ContainerDied","Data":"e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6"} Jul 22 16:09:41.791338 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.790948 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f7ff79bd-p7q4x" event={"ID":"60194dbb-1590-47af-b9da-aa94ffbe0d1f","Type":"ContainerDied","Data":"0ba2465427e89b872153373e570eb7268d47370d306a6cafb7beb1742c3e9df9"} Jul 22 16:09:41.791338 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.790978 2564 scope.go:117] "RemoveContainer" containerID="e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6" Jul 22 16:09:41.801137 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.801110 2564 scope.go:117] "RemoveContainer" containerID="e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6" Jul 22 16:09:41.801414 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:09:41.801390 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6\": container with ID starting with e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6 not found: ID does not exist" containerID="e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6" Jul 22 16:09:41.801478 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.801423 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6"} err="failed to get container status \"e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6\": rpc error: code = NotFound desc = could not find container \"e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6\": container with ID starting with e6a61b8a3743e1ba31e1bcbf49ab62cf61f6595c194ea582910a339fc1dc90c6 not found: ID does not exist" Jul 22 16:09:41.814447 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.814411 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f7ff79bd-p7q4x"] Jul 22 16:09:41.819287 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:41.819256 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64f7ff79bd-p7q4x"] Jul 22 16:09:43.127495 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:09:43.127459 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60194dbb-1590-47af-b9da-aa94ffbe0d1f" path="/var/lib/kubelet/pods/60194dbb-1590-47af-b9da-aa94ffbe0d1f/volumes" Jul 22 16:10:04.361076 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.361045 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 16:10:04.361563 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.361305 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60194dbb-1590-47af-b9da-aa94ffbe0d1f" containerName="console" Jul 22 16:10:04.361563 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.361316 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="60194dbb-1590-47af-b9da-aa94ffbe0d1f" containerName="console" Jul 22 16:10:04.361563 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.361375 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="60194dbb-1590-47af-b9da-aa94ffbe0d1f" containerName="console" Jul 22 16:10:04.364591 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.364574 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.367619 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.367582 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 22 16:10:04.367799 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.367650 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 22 16:10:04.367799 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.367690 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 22 16:10:04.367799 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.367704 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 22 16:10:04.367799 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.367752 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-g27bv\"" Jul 22 16:10:04.367799 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.367696 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 22 16:10:04.367799 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.367582 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 22 16:10:04.368116 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.367652 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 22 16:10:04.368116 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.368010 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 22 16:10:04.377727 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.376028 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 22 16:10:04.380268 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.380239 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 16:10:04.442806 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.442762 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b93bec33-1718-4ba0-ba11-468de38e15e1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.442806 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.442807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-web-config\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443028 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.442827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93bec33-1718-4ba0-ba11-468de38e15e1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443028 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.442910 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fw8d\" (UniqueName: \"kubernetes.io/projected/b93bec33-1718-4ba0-ba11-468de38e15e1-kube-api-access-2fw8d\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443028 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.442944 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b93bec33-1718-4ba0-ba11-468de38e15e1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443028 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.442965 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443028 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.442984 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443028 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.443002 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b93bec33-1718-4ba0-ba11-468de38e15e1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443028 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.443026 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-config-volume\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443236 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.443044 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443236 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.443073 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443236 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.443108 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.443236 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.443131 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b93bec33-1718-4ba0-ba11-468de38e15e1-config-out\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.543925 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.543877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fw8d\" (UniqueName: \"kubernetes.io/projected/b93bec33-1718-4ba0-ba11-468de38e15e1-kube-api-access-2fw8d\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.543925 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.543924 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b93bec33-1718-4ba0-ba11-468de38e15e1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.543944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.543966 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.543988 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b93bec33-1718-4ba0-ba11-468de38e15e1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.544017 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-config-volume\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.544035 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.544064 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.544103 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.544130 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b93bec33-1718-4ba0-ba11-468de38e15e1-config-out\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544531 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.544176 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b93bec33-1718-4ba0-ba11-468de38e15e1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544531 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.544199 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-web-config\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.544531 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.544226 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93bec33-1718-4ba0-ba11-468de38e15e1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.545372 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.545336 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b93bec33-1718-4ba0-ba11-468de38e15e1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.545561 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.545537 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b93bec33-1718-4ba0-ba11-468de38e15e1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.546247 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.546225 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b93bec33-1718-4ba0-ba11-468de38e15e1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.547206 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.547160 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b93bec33-1718-4ba0-ba11-468de38e15e1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.547299 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.547245 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b93bec33-1718-4ba0-ba11-468de38e15e1-config-out\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.547299 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.547276 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.547441 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.547408 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.548032 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.548010 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.548297 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.548262 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-config-volume\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.548393 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.548372 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.548601 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.548581 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-web-config\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.549024 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.549007 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b93bec33-1718-4ba0-ba11-468de38e15e1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.553032 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.553007 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fw8d\" (UniqueName: \"kubernetes.io/projected/b93bec33-1718-4ba0-ba11-468de38e15e1-kube-api-access-2fw8d\") pod \"alertmanager-main-0\" (UID: \"b93bec33-1718-4ba0-ba11-468de38e15e1\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.676461 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.676361 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 22 16:10:04.819585 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.819356 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 16:10:04.822411 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:10:04.822384 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93bec33_1718_4ba0_ba11_468de38e15e1.slice/crio-3967dbead5d8b4018138b52d871e1cac3f67f121bd44b22b03301be15c3fbece WatchSource:0}: Error finding container 3967dbead5d8b4018138b52d871e1cac3f67f121bd44b22b03301be15c3fbece: Status 404 returned error can't find the container with id 3967dbead5d8b4018138b52d871e1cac3f67f121bd44b22b03301be15c3fbece Jul 22 16:10:04.872866 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:04.872826 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b93bec33-1718-4ba0-ba11-468de38e15e1","Type":"ContainerStarted","Data":"3967dbead5d8b4018138b52d871e1cac3f67f121bd44b22b03301be15c3fbece"} Jul 22 16:10:06.882801 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:06.882760 2564 generic.go:356] "Generic (PLEG): container finished" podID="b93bec33-1718-4ba0-ba11-468de38e15e1" containerID="d7dfbf3600499c1f826d8f3d65f7222a27c29c84546f4ed7ee497b5d53246960" exitCode=0 Jul 22 16:10:06.883307 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:06.882832 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b93bec33-1718-4ba0-ba11-468de38e15e1","Type":"ContainerDied","Data":"d7dfbf3600499c1f826d8f3d65f7222a27c29c84546f4ed7ee497b5d53246960"} Jul 22 16:10:08.896282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:08.896242 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b93bec33-1718-4ba0-ba11-468de38e15e1","Type":"ContainerStarted","Data":"2dbec5d58cab7a592701e7e040e821d956c79132f4dcb7c5a9ef19b5bd38ff06"} Jul 22 16:10:08.896745 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:08.896317 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b93bec33-1718-4ba0-ba11-468de38e15e1","Type":"ContainerStarted","Data":"d8a8f7ded3147db378b722ffbb16321099ab3d8f8ed70e2196c23547c6edd700"} Jul 22 16:10:08.896745 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:08.896342 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b93bec33-1718-4ba0-ba11-468de38e15e1","Type":"ContainerStarted","Data":"6d0442d4f6f0349907203f973b466a66565fd6a809a883dce20ab7f540bb7097"} Jul 22 16:10:08.896745 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:08.896367 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b93bec33-1718-4ba0-ba11-468de38e15e1","Type":"ContainerStarted","Data":"7b940342fdc009eb6db97d8e3776f581e3817dc89abd63ee74ac9c5f2917b291"} Jul 22 16:10:09.903321 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:09.903284 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b93bec33-1718-4ba0-ba11-468de38e15e1","Type":"ContainerStarted","Data":"db1d18e23bdc58d14a1c76aa145b74dba57d0cb8de77898d612a206a42984dc8"} Jul 22 16:10:09.903767 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:09.903358 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b93bec33-1718-4ba0-ba11-468de38e15e1","Type":"ContainerStarted","Data":"135c3ae1260d57fbc9faf16dcac8fd279e5d26767b76e87155b05a6a6f2a608d"} Jul 22 16:10:09.964563 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:09.964499 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.827500515 podStartE2EDuration="5.964481065s" totalImagesPullingTime="3.12096021s" totalInitContainerRuntime="16.02034ms" isStatefulPod=false podCreationTimestamp="2026-07-22 16:10:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:10:09.963313575 +0000 UTC m=+173.358855449" watchObservedRunningTime="2026-07-22 16:10:09.964481065 +0000 UTC m=+173.360022899" Jul 22 16:10:11.514104 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.514067 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64d8d54dd4-v6gtf"] Jul 22 16:10:11.516340 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.516321 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.527613 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.527582 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64d8d54dd4-v6gtf"] Jul 22 16:10:11.603668 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.603599 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6rzj\" (UniqueName: \"kubernetes.io/projected/93177ffd-d89e-4312-8c16-fbaca755ec00-kube-api-access-s6rzj\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.603867 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.603725 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-console-config\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.603867 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.603771 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-oauth-config\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.603867 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.603796 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-oauth-serving-cert\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.603867 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.603820 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-serving-cert\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.603867 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.603865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-service-ca\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.604055 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.603883 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-trusted-ca-bundle\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.704830 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.704782 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-oauth-config\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.704830 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.704836 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-oauth-serving-cert\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.705090 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.704958 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-serving-cert\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.705090 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.705004 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-service-ca\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.705090 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.705031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-trusted-ca-bundle\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.705090 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.705085 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6rzj\" (UniqueName: \"kubernetes.io/projected/93177ffd-d89e-4312-8c16-fbaca755ec00-kube-api-access-s6rzj\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.705291 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.705155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-console-config\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.705609 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.705587 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-oauth-serving-cert\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.705727 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.705674 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-service-ca\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.705877 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.705856 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-console-config\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.705968 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.705944 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-trusted-ca-bundle\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.707587 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.707560 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-oauth-config\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.707587 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.707574 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-serving-cert\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.714376 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.714348 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6rzj\" (UniqueName: \"kubernetes.io/projected/93177ffd-d89e-4312-8c16-fbaca755ec00-kube-api-access-s6rzj\") pod \"console-64d8d54dd4-v6gtf\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.825724 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.825608 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:11.963681 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:11.963503 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64d8d54dd4-v6gtf"] Jul 22 16:10:11.965744 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:10:11.965709 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93177ffd_d89e_4312_8c16_fbaca755ec00.slice/crio-e69a7969340456fe4bed5f75cbbbb9067471b47ae7333d178a9a59889b257e09 WatchSource:0}: Error finding container e69a7969340456fe4bed5f75cbbbb9067471b47ae7333d178a9a59889b257e09: Status 404 returned error can't find the container with id e69a7969340456fe4bed5f75cbbbb9067471b47ae7333d178a9a59889b257e09 Jul 22 16:10:12.919295 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:12.919246 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d8d54dd4-v6gtf" event={"ID":"93177ffd-d89e-4312-8c16-fbaca755ec00","Type":"ContainerStarted","Data":"55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80"} Jul 22 16:10:12.919732 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:12.919311 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d8d54dd4-v6gtf" event={"ID":"93177ffd-d89e-4312-8c16-fbaca755ec00","Type":"ContainerStarted","Data":"e69a7969340456fe4bed5f75cbbbb9067471b47ae7333d178a9a59889b257e09"} Jul 22 16:10:12.940069 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:12.940009 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64d8d54dd4-v6gtf" podStartSLOduration=1.939991289 podStartE2EDuration="1.939991289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:10:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:10:12.939612208 +0000 UTC m=+176.335154041" watchObservedRunningTime="2026-07-22 16:10:12.939991289 +0000 UTC m=+176.335533230" Jul 22 16:10:21.826700 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:21.826618 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:21.827342 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:21.826739 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:21.831848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:21.831817 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:21.959253 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:21.959222 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:10:22.000995 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:22.000948 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5455985cf7-z8x72"] Jul 22 16:10:47.023794 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.023752 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5455985cf7-z8x72" podUID="af6f1cc7-7692-43b5-b27e-c0f66584db77" containerName="console" containerID="cri-o://5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5" gracePeriod=15 Jul 22 16:10:47.267154 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.267125 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:10:47.310204 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310123 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-config\") pod \"af6f1cc7-7692-43b5-b27e-c0f66584db77\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " Jul 22 16:10:47.310204 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310198 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-oauth-config\") pod \"af6f1cc7-7692-43b5-b27e-c0f66584db77\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " Jul 22 16:10:47.310436 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310230 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-oauth-serving-cert\") pod \"af6f1cc7-7692-43b5-b27e-c0f66584db77\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " Jul 22 16:10:47.310436 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310255 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-serving-cert\") pod \"af6f1cc7-7692-43b5-b27e-c0f66584db77\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " Jul 22 16:10:47.310436 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310315 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-service-ca\") pod \"af6f1cc7-7692-43b5-b27e-c0f66584db77\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " Jul 22 16:10:47.310436 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310344 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgs4g\" (UniqueName: \"kubernetes.io/projected/af6f1cc7-7692-43b5-b27e-c0f66584db77-kube-api-access-rgs4g\") pod \"af6f1cc7-7692-43b5-b27e-c0f66584db77\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " Jul 22 16:10:47.310436 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310366 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-trusted-ca-bundle\") pod \"af6f1cc7-7692-43b5-b27e-c0f66584db77\" (UID: \"af6f1cc7-7692-43b5-b27e-c0f66584db77\") " Jul 22 16:10:47.310717 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310497 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-config" (OuterVolumeSpecName: "console-config") pod "af6f1cc7-7692-43b5-b27e-c0f66584db77" (UID: "af6f1cc7-7692-43b5-b27e-c0f66584db77"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:10:47.310717 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310668 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "af6f1cc7-7692-43b5-b27e-c0f66584db77" (UID: "af6f1cc7-7692-43b5-b27e-c0f66584db77"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:10:47.310793 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310704 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-service-ca" (OuterVolumeSpecName: "service-ca") pod "af6f1cc7-7692-43b5-b27e-c0f66584db77" (UID: "af6f1cc7-7692-43b5-b27e-c0f66584db77"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:10:47.310969 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.310945 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "af6f1cc7-7692-43b5-b27e-c0f66584db77" (UID: "af6f1cc7-7692-43b5-b27e-c0f66584db77"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:10:47.312478 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.312450 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "af6f1cc7-7692-43b5-b27e-c0f66584db77" (UID: "af6f1cc7-7692-43b5-b27e-c0f66584db77"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:10:47.312583 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.312517 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af6f1cc7-7692-43b5-b27e-c0f66584db77-kube-api-access-rgs4g" (OuterVolumeSpecName: "kube-api-access-rgs4g") pod "af6f1cc7-7692-43b5-b27e-c0f66584db77" (UID: "af6f1cc7-7692-43b5-b27e-c0f66584db77"). InnerVolumeSpecName "kube-api-access-rgs4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:10:47.312583 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.312542 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "af6f1cc7-7692-43b5-b27e-c0f66584db77" (UID: "af6f1cc7-7692-43b5-b27e-c0f66584db77"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:10:47.411833 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.411795 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-service-ca\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:10:47.411833 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.411827 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgs4g\" (UniqueName: \"kubernetes.io/projected/af6f1cc7-7692-43b5-b27e-c0f66584db77-kube-api-access-rgs4g\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:10:47.411833 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.411838 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-trusted-ca-bundle\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:10:47.411833 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.411847 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-config\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:10:47.412109 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.411856 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-oauth-config\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:10:47.412109 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.411865 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af6f1cc7-7692-43b5-b27e-c0f66584db77-oauth-serving-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:10:47.412109 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:47.411873 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af6f1cc7-7692-43b5-b27e-c0f66584db77-console-serving-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:10:48.056371 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:48.056336 2564 generic.go:356] "Generic (PLEG): container finished" podID="af6f1cc7-7692-43b5-b27e-c0f66584db77" containerID="5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5" exitCode=0 Jul 22 16:10:48.056848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:48.056387 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5455985cf7-z8x72" Jul 22 16:10:48.056848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:48.056394 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5455985cf7-z8x72" event={"ID":"af6f1cc7-7692-43b5-b27e-c0f66584db77","Type":"ContainerDied","Data":"5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5"} Jul 22 16:10:48.056848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:48.056436 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5455985cf7-z8x72" event={"ID":"af6f1cc7-7692-43b5-b27e-c0f66584db77","Type":"ContainerDied","Data":"cafc5a18668ff482804abcdf25a4119e0a61579172ced5db041549ed1d532364"} Jul 22 16:10:48.056848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:48.056455 2564 scope.go:117] "RemoveContainer" containerID="5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5" Jul 22 16:10:48.065025 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:48.065008 2564 scope.go:117] "RemoveContainer" containerID="5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5" Jul 22 16:10:48.065283 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:10:48.065264 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5\": container with ID starting with 5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5 not found: ID does not exist" containerID="5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5" Jul 22 16:10:48.065320 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:48.065295 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5"} err="failed to get container status \"5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5\": rpc error: code = NotFound desc = could not find container \"5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5\": container with ID starting with 5b296f774bf12b21cc8fdd453e235f5caaf45c9af7b88326617a7ac2c99bcef5 not found: ID does not exist" Jul 22 16:10:48.080535 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:48.080505 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5455985cf7-z8x72"] Jul 22 16:10:48.090252 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:48.090226 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5455985cf7-z8x72"] Jul 22 16:10:49.127131 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:10:49.127095 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af6f1cc7-7692-43b5-b27e-c0f66584db77" path="/var/lib/kubelet/pods/af6f1cc7-7692-43b5-b27e-c0f66584db77/volumes" Jul 22 16:11:22.780583 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.780538 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f55d47cbf-kz2nq"] Jul 22 16:11:22.781188 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.780865 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af6f1cc7-7692-43b5-b27e-c0f66584db77" containerName="console" Jul 22 16:11:22.781188 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.780882 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="af6f1cc7-7692-43b5-b27e-c0f66584db77" containerName="console" Jul 22 16:11:22.781188 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.780954 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="af6f1cc7-7692-43b5-b27e-c0f66584db77" containerName="console" Jul 22 16:11:22.785379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.785358 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.798142 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.798112 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f55d47cbf-kz2nq"] Jul 22 16:11:22.892321 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.892284 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-oauth-serving-cert\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.892321 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.892323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-oauth-config\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.892563 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.892343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-service-ca\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.892563 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.892439 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-serving-cert\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.892563 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.892468 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-trusted-ca-bundle\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.892563 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.892489 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-config\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.892563 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.892513 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5kx8\" (UniqueName: \"kubernetes.io/projected/86d1e78c-ade2-43ed-a901-77c4daac75d7-kube-api-access-k5kx8\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.993417 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.993380 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5kx8\" (UniqueName: \"kubernetes.io/projected/86d1e78c-ade2-43ed-a901-77c4daac75d7-kube-api-access-k5kx8\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.993659 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.993434 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-oauth-serving-cert\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.993659 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.993461 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-oauth-config\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.993659 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.993479 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-service-ca\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.993659 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.993505 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-serving-cert\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.993659 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.993535 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-trusted-ca-bundle\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.993927 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.993748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-config\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.994322 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.994297 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-oauth-serving-cert\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.994430 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.994408 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-service-ca\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.994511 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.994495 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-trusted-ca-bundle\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.994548 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.994502 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-config\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.996042 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.996011 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-oauth-config\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:22.996182 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:22.996161 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-serving-cert\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:23.003173 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:23.003152 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5kx8\" (UniqueName: \"kubernetes.io/projected/86d1e78c-ade2-43ed-a901-77c4daac75d7-kube-api-access-k5kx8\") pod \"console-5f55d47cbf-kz2nq\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:23.094239 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:23.094135 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:23.227202 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:23.227174 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f55d47cbf-kz2nq"] Jul 22 16:11:23.229947 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:11:23.229921 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d1e78c_ade2_43ed_a901_77c4daac75d7.slice/crio-787e211684c444385c36f35e574269908d9ba1a0676bd59313d04b31d3191d18 WatchSource:0}: Error finding container 787e211684c444385c36f35e574269908d9ba1a0676bd59313d04b31d3191d18: Status 404 returned error can't find the container with id 787e211684c444385c36f35e574269908d9ba1a0676bd59313d04b31d3191d18 Jul 22 16:11:24.194097 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:24.194060 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f55d47cbf-kz2nq" event={"ID":"86d1e78c-ade2-43ed-a901-77c4daac75d7","Type":"ContainerStarted","Data":"2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49"} Jul 22 16:11:24.194514 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:24.194116 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f55d47cbf-kz2nq" event={"ID":"86d1e78c-ade2-43ed-a901-77c4daac75d7","Type":"ContainerStarted","Data":"787e211684c444385c36f35e574269908d9ba1a0676bd59313d04b31d3191d18"} Jul 22 16:11:24.216494 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:24.216429 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f55d47cbf-kz2nq" podStartSLOduration=2.216414726 podStartE2EDuration="2.216414726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:11:24.215696451 +0000 UTC m=+247.611238293" watchObservedRunningTime="2026-07-22 16:11:24.216414726 +0000 UTC m=+247.611956559" Jul 22 16:11:33.094787 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:33.094741 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:33.095304 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:33.094811 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:33.099604 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:33.099580 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:33.231810 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:33.231779 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:11:33.299327 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:33.299293 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64d8d54dd4-v6gtf"] Jul 22 16:11:58.320004 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.319944 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64d8d54dd4-v6gtf" podUID="93177ffd-d89e-4312-8c16-fbaca755ec00" containerName="console" containerID="cri-o://55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80" gracePeriod=15 Jul 22 16:11:58.564903 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.564876 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:11:58.584617 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.584549 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-oauth-config\") pod \"93177ffd-d89e-4312-8c16-fbaca755ec00\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " Jul 22 16:11:58.584617 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.584581 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-trusted-ca-bundle\") pod \"93177ffd-d89e-4312-8c16-fbaca755ec00\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " Jul 22 16:11:58.584617 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.584602 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-serving-cert\") pod \"93177ffd-d89e-4312-8c16-fbaca755ec00\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " Jul 22 16:11:58.584869 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.584637 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-oauth-serving-cert\") pod \"93177ffd-d89e-4312-8c16-fbaca755ec00\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " Jul 22 16:11:58.584869 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.584777 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-console-config\") pod \"93177ffd-d89e-4312-8c16-fbaca755ec00\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " Jul 22 16:11:58.584869 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.584823 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6rzj\" (UniqueName: \"kubernetes.io/projected/93177ffd-d89e-4312-8c16-fbaca755ec00-kube-api-access-s6rzj\") pod \"93177ffd-d89e-4312-8c16-fbaca755ec00\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " Jul 22 16:11:58.584869 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.584854 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-service-ca\") pod \"93177ffd-d89e-4312-8c16-fbaca755ec00\" (UID: \"93177ffd-d89e-4312-8c16-fbaca755ec00\") " Jul 22 16:11:58.585114 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.585062 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "93177ffd-d89e-4312-8c16-fbaca755ec00" (UID: "93177ffd-d89e-4312-8c16-fbaca755ec00"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:11:58.585114 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.585073 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "93177ffd-d89e-4312-8c16-fbaca755ec00" (UID: "93177ffd-d89e-4312-8c16-fbaca755ec00"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:11:58.585305 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.585130 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-console-config" (OuterVolumeSpecName: "console-config") pod "93177ffd-d89e-4312-8c16-fbaca755ec00" (UID: "93177ffd-d89e-4312-8c16-fbaca755ec00"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:11:58.585305 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.585173 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-trusted-ca-bundle\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:11:58.585305 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.585187 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-oauth-serving-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:11:58.585447 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.585355 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-service-ca" (OuterVolumeSpecName: "service-ca") pod "93177ffd-d89e-4312-8c16-fbaca755ec00" (UID: "93177ffd-d89e-4312-8c16-fbaca755ec00"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:11:58.586867 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.586844 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "93177ffd-d89e-4312-8c16-fbaca755ec00" (UID: "93177ffd-d89e-4312-8c16-fbaca755ec00"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:11:58.587203 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.587180 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93177ffd-d89e-4312-8c16-fbaca755ec00-kube-api-access-s6rzj" (OuterVolumeSpecName: "kube-api-access-s6rzj") pod "93177ffd-d89e-4312-8c16-fbaca755ec00" (UID: "93177ffd-d89e-4312-8c16-fbaca755ec00"). InnerVolumeSpecName "kube-api-access-s6rzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:11:58.587389 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.587288 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "93177ffd-d89e-4312-8c16-fbaca755ec00" (UID: "93177ffd-d89e-4312-8c16-fbaca755ec00"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:11:58.686082 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.686039 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-console-config\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:11:58.686082 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.686072 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6rzj\" (UniqueName: \"kubernetes.io/projected/93177ffd-d89e-4312-8c16-fbaca755ec00-kube-api-access-s6rzj\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:11:58.686082 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.686087 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93177ffd-d89e-4312-8c16-fbaca755ec00-service-ca\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:11:58.686329 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.686099 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-oauth-config\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:11:58.686329 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:58.686113 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93177ffd-d89e-4312-8c16-fbaca755ec00-console-serving-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:11:59.325088 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:59.325046 2564 generic.go:356] "Generic (PLEG): container finished" podID="93177ffd-d89e-4312-8c16-fbaca755ec00" containerID="55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80" exitCode=0 Jul 22 16:11:59.325519 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:59.325096 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d8d54dd4-v6gtf" Jul 22 16:11:59.325519 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:59.325130 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d8d54dd4-v6gtf" event={"ID":"93177ffd-d89e-4312-8c16-fbaca755ec00","Type":"ContainerDied","Data":"55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80"} Jul 22 16:11:59.325519 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:59.325186 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d8d54dd4-v6gtf" event={"ID":"93177ffd-d89e-4312-8c16-fbaca755ec00","Type":"ContainerDied","Data":"e69a7969340456fe4bed5f75cbbbb9067471b47ae7333d178a9a59889b257e09"} Jul 22 16:11:59.325519 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:59.325209 2564 scope.go:117] "RemoveContainer" containerID="55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80" Jul 22 16:11:59.333383 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:59.333365 2564 scope.go:117] "RemoveContainer" containerID="55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80" Jul 22 16:11:59.333682 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:11:59.333663 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80\": container with ID starting with 55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80 not found: ID does not exist" containerID="55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80" Jul 22 16:11:59.333758 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:59.333690 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80"} err="failed to get container status \"55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80\": rpc error: code = NotFound desc = could not find container \"55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80\": container with ID starting with 55b07612fe3456711a6e8a40559e932c6b0289b358c8d566d5387e4934a6eb80 not found: ID does not exist" Jul 22 16:11:59.347144 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:59.347121 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64d8d54dd4-v6gtf"] Jul 22 16:11:59.351382 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:11:59.351360 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64d8d54dd4-v6gtf"] Jul 22 16:12:01.127782 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:01.127746 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93177ffd-d89e-4312-8c16-fbaca755ec00" path="/var/lib/kubelet/pods/93177ffd-d89e-4312-8c16-fbaca755ec00/volumes" Jul 22 16:12:16.988059 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:16.988027 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 16:12:26.009134 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.009095 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg"] Jul 22 16:12:26.011403 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.009388 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93177ffd-d89e-4312-8c16-fbaca755ec00" containerName="console" Jul 22 16:12:26.011403 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.009398 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="93177ffd-d89e-4312-8c16-fbaca755ec00" containerName="console" Jul 22 16:12:26.011403 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.009456 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="93177ffd-d89e-4312-8c16-fbaca755ec00" containerName="console" Jul 22 16:12:26.012293 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.012276 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.018356 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.018323 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 16:12:26.018939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.018706 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wgdqm\"" Jul 22 16:12:26.019859 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.019837 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 16:12:26.043297 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.043264 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg"] Jul 22 16:12:26.115953 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.115907 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.116154 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.115978 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk7vn\" (UniqueName: \"kubernetes.io/projected/3ada060d-e2f6-41fc-8017-9789041bce84-kube-api-access-jk7vn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.116154 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.116029 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.216829 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.216789 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.217004 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.216847 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.217004 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.216889 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk7vn\" (UniqueName: \"kubernetes.io/projected/3ada060d-e2f6-41fc-8017-9789041bce84-kube-api-access-jk7vn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.217201 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.217181 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.217308 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.217288 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.229587 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.229547 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk7vn\" (UniqueName: \"kubernetes.io/projected/3ada060d-e2f6-41fc-8017-9789041bce84-kube-api-access-jk7vn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.321320 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.321231 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:26.468976 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.468942 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg"] Jul 22 16:12:26.472678 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:12:26.472641 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ada060d_e2f6_41fc_8017_9789041bce84.slice/crio-2344c4d495418a5def3fd89afff7470ef92c5d5786d58c6046d0bfe960e7aee7 WatchSource:0}: Error finding container 2344c4d495418a5def3fd89afff7470ef92c5d5786d58c6046d0bfe960e7aee7: Status 404 returned error can't find the container with id 2344c4d495418a5def3fd89afff7470ef92c5d5786d58c6046d0bfe960e7aee7 Jul 22 16:12:26.474769 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:26.474751 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:12:27.431046 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:27.430997 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" event={"ID":"3ada060d-e2f6-41fc-8017-9789041bce84","Type":"ContainerStarted","Data":"2344c4d495418a5def3fd89afff7470ef92c5d5786d58c6046d0bfe960e7aee7"} Jul 22 16:12:34.461062 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:34.461022 2564 generic.go:356] "Generic (PLEG): container finished" podID="3ada060d-e2f6-41fc-8017-9789041bce84" containerID="e620edcac23b9ddac70b5011f050ab5237ab312cf03d8bf351431efc1f58d655" exitCode=0 Jul 22 16:12:34.461475 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:34.461083 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" event={"ID":"3ada060d-e2f6-41fc-8017-9789041bce84","Type":"ContainerDied","Data":"e620edcac23b9ddac70b5011f050ab5237ab312cf03d8bf351431efc1f58d655"} Jul 22 16:12:37.475636 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:37.475597 2564 generic.go:356] "Generic (PLEG): container finished" podID="3ada060d-e2f6-41fc-8017-9789041bce84" containerID="1943b978c0ae4a412a99ec9bbafaab294a68cf7317d708d027597b2e8f1eab17" exitCode=0 Jul 22 16:12:37.476038 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:37.475678 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" event={"ID":"3ada060d-e2f6-41fc-8017-9789041bce84","Type":"ContainerDied","Data":"1943b978c0ae4a412a99ec9bbafaab294a68cf7317d708d027597b2e8f1eab17"} Jul 22 16:12:46.514483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:46.514442 2564 generic.go:356] "Generic (PLEG): container finished" podID="3ada060d-e2f6-41fc-8017-9789041bce84" containerID="b58ea13f3172d64d5f34a81e39c1260ac93ed40c632f9e97f69e3fbc7060c5d7" exitCode=0 Jul 22 16:12:46.514968 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:46.514519 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" event={"ID":"3ada060d-e2f6-41fc-8017-9789041bce84","Type":"ContainerDied","Data":"b58ea13f3172d64d5f34a81e39c1260ac93ed40c632f9e97f69e3fbc7060c5d7"} Jul 22 16:12:47.643909 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:47.643883 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:47.701119 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:47.701083 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-bundle\") pod \"3ada060d-e2f6-41fc-8017-9789041bce84\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " Jul 22 16:12:47.701375 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:47.701130 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-util\") pod \"3ada060d-e2f6-41fc-8017-9789041bce84\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " Jul 22 16:12:47.701375 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:47.701163 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk7vn\" (UniqueName: \"kubernetes.io/projected/3ada060d-e2f6-41fc-8017-9789041bce84-kube-api-access-jk7vn\") pod \"3ada060d-e2f6-41fc-8017-9789041bce84\" (UID: \"3ada060d-e2f6-41fc-8017-9789041bce84\") " Jul 22 16:12:47.701739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:47.701712 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-bundle" (OuterVolumeSpecName: "bundle") pod "3ada060d-e2f6-41fc-8017-9789041bce84" (UID: "3ada060d-e2f6-41fc-8017-9789041bce84"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:12:47.703425 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:47.703398 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ada060d-e2f6-41fc-8017-9789041bce84-kube-api-access-jk7vn" (OuterVolumeSpecName: "kube-api-access-jk7vn") pod "3ada060d-e2f6-41fc-8017-9789041bce84" (UID: "3ada060d-e2f6-41fc-8017-9789041bce84"). InnerVolumeSpecName "kube-api-access-jk7vn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:12:47.706237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:47.706217 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-util" (OuterVolumeSpecName: "util") pod "3ada060d-e2f6-41fc-8017-9789041bce84" (UID: "3ada060d-e2f6-41fc-8017-9789041bce84"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:12:47.801883 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:47.801791 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-bundle\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:12:47.801883 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:47.801820 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ada060d-e2f6-41fc-8017-9789041bce84-util\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:12:47.801883 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:47.801830 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk7vn\" (UniqueName: \"kubernetes.io/projected/3ada060d-e2f6-41fc-8017-9789041bce84-kube-api-access-jk7vn\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:12:48.523162 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:48.523126 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" Jul 22 16:12:48.523344 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:48.523125 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjnmg" event={"ID":"3ada060d-e2f6-41fc-8017-9789041bce84","Type":"ContainerDied","Data":"2344c4d495418a5def3fd89afff7470ef92c5d5786d58c6046d0bfe960e7aee7"} Jul 22 16:12:48.523344 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:12:48.523249 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2344c4d495418a5def3fd89afff7470ef92c5d5786d58c6046d0bfe960e7aee7" Jul 22 16:13:00.622599 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.622550 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w"] Jul 22 16:13:00.623150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.622885 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ada060d-e2f6-41fc-8017-9789041bce84" containerName="extract" Jul 22 16:13:00.623150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.622897 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ada060d-e2f6-41fc-8017-9789041bce84" containerName="extract" Jul 22 16:13:00.623150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.622910 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ada060d-e2f6-41fc-8017-9789041bce84" containerName="util" Jul 22 16:13:00.623150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.622915 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ada060d-e2f6-41fc-8017-9789041bce84" containerName="util" Jul 22 16:13:00.623150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.622925 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ada060d-e2f6-41fc-8017-9789041bce84" containerName="pull" Jul 22 16:13:00.623150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.622931 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ada060d-e2f6-41fc-8017-9789041bce84" containerName="pull" Jul 22 16:13:00.623150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.622985 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ada060d-e2f6-41fc-8017-9789041bce84" containerName="extract" Jul 22 16:13:00.625251 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.625234 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:00.628304 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.628277 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 16:13:00.628462 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.628437 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wgdqm\"" Jul 22 16:13:00.629286 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.629265 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 16:13:00.654998 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.654964 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w"] Jul 22 16:13:00.697299 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.697263 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:00.697473 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.697310 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9qsg\" (UniqueName: \"kubernetes.io/projected/a781ee54-68cf-4d63-a68d-8934d6f27237-kube-api-access-n9qsg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:00.697473 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.697351 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:00.798646 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.798590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:00.798797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.798666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9qsg\" (UniqueName: \"kubernetes.io/projected/a781ee54-68cf-4d63-a68d-8934d6f27237-kube-api-access-n9qsg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:00.798797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.798694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:00.798987 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.798970 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:00.799025 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.798997 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:00.843769 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.843735 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9qsg\" (UniqueName: \"kubernetes.io/projected/a781ee54-68cf-4d63-a68d-8934d6f27237-kube-api-access-n9qsg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:00.934881 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:00.934787 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:01.125847 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:13:01.125810 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda781ee54_68cf_4d63_a68d_8934d6f27237.slice/crio-6e68f42b83db6fcde5c7cdb63395635d8329ad1c1f7385eb3014c68fc73834e8 WatchSource:0}: Error finding container 6e68f42b83db6fcde5c7cdb63395635d8329ad1c1f7385eb3014c68fc73834e8: Status 404 returned error can't find the container with id 6e68f42b83db6fcde5c7cdb63395635d8329ad1c1f7385eb3014c68fc73834e8 Jul 22 16:13:01.127571 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:01.127540 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w"] Jul 22 16:13:01.573813 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:01.573781 2564 generic.go:356] "Generic (PLEG): container finished" podID="a781ee54-68cf-4d63-a68d-8934d6f27237" containerID="d412c655d16d93aaa5ad8d8e560d1cd53f95f8b01b0ab5b33054587818abc0a7" exitCode=0 Jul 22 16:13:01.573973 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:01.573868 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" event={"ID":"a781ee54-68cf-4d63-a68d-8934d6f27237","Type":"ContainerDied","Data":"d412c655d16d93aaa5ad8d8e560d1cd53f95f8b01b0ab5b33054587818abc0a7"} Jul 22 16:13:01.573973 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:01.573926 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" event={"ID":"a781ee54-68cf-4d63-a68d-8934d6f27237","Type":"ContainerStarted","Data":"6e68f42b83db6fcde5c7cdb63395635d8329ad1c1f7385eb3014c68fc73834e8"} Jul 22 16:13:03.401382 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.401338 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl"] Jul 22 16:13:03.405911 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.405888 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" Jul 22 16:13:03.409092 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.409064 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 16:13:03.409921 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.409901 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2tsll\"" Jul 22 16:13:03.410023 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.409952 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 16:13:03.419433 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.419407 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bjpf\" (UniqueName: \"kubernetes.io/projected/1d3af409-80eb-491b-9e56-fb091e7b4372-kube-api-access-8bjpf\") pod \"cert-manager-cainjector-559cdd8c87-sjvvl\" (UID: \"1d3af409-80eb-491b-9e56-fb091e7b4372\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" Jul 22 16:13:03.419545 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.419444 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d3af409-80eb-491b-9e56-fb091e7b4372-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sjvvl\" (UID: \"1d3af409-80eb-491b-9e56-fb091e7b4372\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" Jul 22 16:13:03.448754 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.448720 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl"] Jul 22 16:13:03.520198 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.519941 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bjpf\" (UniqueName: \"kubernetes.io/projected/1d3af409-80eb-491b-9e56-fb091e7b4372-kube-api-access-8bjpf\") pod \"cert-manager-cainjector-559cdd8c87-sjvvl\" (UID: \"1d3af409-80eb-491b-9e56-fb091e7b4372\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" Jul 22 16:13:03.520198 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.519991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d3af409-80eb-491b-9e56-fb091e7b4372-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sjvvl\" (UID: \"1d3af409-80eb-491b-9e56-fb091e7b4372\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" Jul 22 16:13:03.532116 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.532081 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bjpf\" (UniqueName: \"kubernetes.io/projected/1d3af409-80eb-491b-9e56-fb091e7b4372-kube-api-access-8bjpf\") pod \"cert-manager-cainjector-559cdd8c87-sjvvl\" (UID: \"1d3af409-80eb-491b-9e56-fb091e7b4372\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" Jul 22 16:13:03.533957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.533931 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d3af409-80eb-491b-9e56-fb091e7b4372-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-sjvvl\" (UID: \"1d3af409-80eb-491b-9e56-fb091e7b4372\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" Jul 22 16:13:03.717067 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.716972 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" Jul 22 16:13:03.947547 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:03.947504 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl"] Jul 22 16:13:03.979488 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:13:03.979454 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d3af409_80eb_491b_9e56_fb091e7b4372.slice/crio-32826293265f556f80585f08d7a4b6ee846ce656477d17a8aa4a9f6bf8f1fc60 WatchSource:0}: Error finding container 32826293265f556f80585f08d7a4b6ee846ce656477d17a8aa4a9f6bf8f1fc60: Status 404 returned error can't find the container with id 32826293265f556f80585f08d7a4b6ee846ce656477d17a8aa4a9f6bf8f1fc60 Jul 22 16:13:04.586919 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:04.586882 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" event={"ID":"1d3af409-80eb-491b-9e56-fb091e7b4372","Type":"ContainerStarted","Data":"32826293265f556f80585f08d7a4b6ee846ce656477d17a8aa4a9f6bf8f1fc60"} Jul 22 16:13:04.588688 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:04.588659 2564 generic.go:356] "Generic (PLEG): container finished" podID="a781ee54-68cf-4d63-a68d-8934d6f27237" containerID="133244ac560d81aa50595cd028cb390831991ef87ef2dedda2ea8f0f6f569622" exitCode=0 Jul 22 16:13:04.588811 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:04.588701 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" event={"ID":"a781ee54-68cf-4d63-a68d-8934d6f27237","Type":"ContainerDied","Data":"133244ac560d81aa50595cd028cb390831991ef87ef2dedda2ea8f0f6f569622"} Jul 22 16:13:05.595891 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:05.595853 2564 generic.go:356] "Generic (PLEG): container finished" podID="a781ee54-68cf-4d63-a68d-8934d6f27237" containerID="1837e321c75eb8e331097dfd0685f69bdc1d3bfc546cdd0c4b7a3bee43e2f088" exitCode=0 Jul 22 16:13:05.596327 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:05.595907 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" event={"ID":"a781ee54-68cf-4d63-a68d-8934d6f27237","Type":"ContainerDied","Data":"1837e321c75eb8e331097dfd0685f69bdc1d3bfc546cdd0c4b7a3bee43e2f088"} Jul 22 16:13:06.742918 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:06.742890 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:06.850476 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:06.850413 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-util\") pod \"a781ee54-68cf-4d63-a68d-8934d6f27237\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " Jul 22 16:13:06.850603 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:06.850489 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-bundle\") pod \"a781ee54-68cf-4d63-a68d-8934d6f27237\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " Jul 22 16:13:06.850603 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:06.850510 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9qsg\" (UniqueName: \"kubernetes.io/projected/a781ee54-68cf-4d63-a68d-8934d6f27237-kube-api-access-n9qsg\") pod \"a781ee54-68cf-4d63-a68d-8934d6f27237\" (UID: \"a781ee54-68cf-4d63-a68d-8934d6f27237\") " Jul 22 16:13:06.850872 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:06.850839 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-bundle" (OuterVolumeSpecName: "bundle") pod "a781ee54-68cf-4d63-a68d-8934d6f27237" (UID: "a781ee54-68cf-4d63-a68d-8934d6f27237"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:13:06.852747 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:06.852724 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a781ee54-68cf-4d63-a68d-8934d6f27237-kube-api-access-n9qsg" (OuterVolumeSpecName: "kube-api-access-n9qsg") pod "a781ee54-68cf-4d63-a68d-8934d6f27237" (UID: "a781ee54-68cf-4d63-a68d-8934d6f27237"). InnerVolumeSpecName "kube-api-access-n9qsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:13:06.855054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:06.855018 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-util" (OuterVolumeSpecName: "util") pod "a781ee54-68cf-4d63-a68d-8934d6f27237" (UID: "a781ee54-68cf-4d63-a68d-8934d6f27237"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:13:06.951517 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:06.951485 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-bundle\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:13:06.951517 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:06.951513 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9qsg\" (UniqueName: \"kubernetes.io/projected/a781ee54-68cf-4d63-a68d-8934d6f27237-kube-api-access-n9qsg\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:13:06.951517 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:06.951521 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a781ee54-68cf-4d63-a68d-8934d6f27237-util\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:13:07.604544 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:07.604504 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" event={"ID":"1d3af409-80eb-491b-9e56-fb091e7b4372","Type":"ContainerStarted","Data":"d96dd88d13375b34ce17c50d9effa10e59393e7150de10cea0233844da32e01f"} Jul 22 16:13:07.606908 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:07.606876 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" event={"ID":"a781ee54-68cf-4d63-a68d-8934d6f27237","Type":"ContainerDied","Data":"6e68f42b83db6fcde5c7cdb63395635d8329ad1c1f7385eb3014c68fc73834e8"} Jul 22 16:13:07.607068 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:07.606918 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e68f42b83db6fcde5c7cdb63395635d8329ad1c1f7385eb3014c68fc73834e8" Jul 22 16:13:07.607068 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:07.606936 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh866w" Jul 22 16:13:07.875187 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:07.875077 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-sjvvl" podStartSLOduration=2.034018 podStartE2EDuration="4.875060579s" totalImagesPullingTime="2.841042579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:07.642696704 +0000 UTC m=+351.038238537" watchObservedRunningTime="2026-07-22 16:13:07.875060579 +0000 UTC m=+351.270602413" Jul 22 16:13:10.323800 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.323704 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wqmg9"] Jul 22 16:13:10.324175 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.324078 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a781ee54-68cf-4d63-a68d-8934d6f27237" containerName="util" Jul 22 16:13:10.324175 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.324094 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a781ee54-68cf-4d63-a68d-8934d6f27237" containerName="util" Jul 22 16:13:10.324175 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.324107 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a781ee54-68cf-4d63-a68d-8934d6f27237" containerName="extract" Jul 22 16:13:10.324175 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.324114 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a781ee54-68cf-4d63-a68d-8934d6f27237" containerName="extract" Jul 22 16:13:10.324175 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.324131 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a781ee54-68cf-4d63-a68d-8934d6f27237" containerName="pull" Jul 22 16:13:10.324175 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.324137 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a781ee54-68cf-4d63-a68d-8934d6f27237" containerName="pull" Jul 22 16:13:10.324386 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.324193 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a781ee54-68cf-4d63-a68d-8934d6f27237" containerName="extract" Jul 22 16:13:10.326013 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.325997 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wqmg9" Jul 22 16:13:10.330744 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.330724 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5zhrk\"" Jul 22 16:13:10.380615 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.380581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vntld\" (UniqueName: \"kubernetes.io/projected/3ba6351b-0b73-46b6-8d25-b5a3df8f2507-kube-api-access-vntld\") pod \"cert-manager-55b4d954d5-wqmg9\" (UID: \"3ba6351b-0b73-46b6-8d25-b5a3df8f2507\") " pod="cert-manager/cert-manager-55b4d954d5-wqmg9" Jul 22 16:13:10.380823 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.380640 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ba6351b-0b73-46b6-8d25-b5a3df8f2507-bound-sa-token\") pod \"cert-manager-55b4d954d5-wqmg9\" (UID: \"3ba6351b-0b73-46b6-8d25-b5a3df8f2507\") " pod="cert-manager/cert-manager-55b4d954d5-wqmg9" Jul 22 16:13:10.388336 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.388304 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wqmg9"] Jul 22 16:13:10.481192 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.481157 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vntld\" (UniqueName: \"kubernetes.io/projected/3ba6351b-0b73-46b6-8d25-b5a3df8f2507-kube-api-access-vntld\") pod \"cert-manager-55b4d954d5-wqmg9\" (UID: \"3ba6351b-0b73-46b6-8d25-b5a3df8f2507\") " pod="cert-manager/cert-manager-55b4d954d5-wqmg9" Jul 22 16:13:10.481366 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.481209 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ba6351b-0b73-46b6-8d25-b5a3df8f2507-bound-sa-token\") pod \"cert-manager-55b4d954d5-wqmg9\" (UID: \"3ba6351b-0b73-46b6-8d25-b5a3df8f2507\") " pod="cert-manager/cert-manager-55b4d954d5-wqmg9" Jul 22 16:13:10.520864 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.520827 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3ba6351b-0b73-46b6-8d25-b5a3df8f2507-bound-sa-token\") pod \"cert-manager-55b4d954d5-wqmg9\" (UID: \"3ba6351b-0b73-46b6-8d25-b5a3df8f2507\") " pod="cert-manager/cert-manager-55b4d954d5-wqmg9" Jul 22 16:13:10.521672 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.521617 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vntld\" (UniqueName: \"kubernetes.io/projected/3ba6351b-0b73-46b6-8d25-b5a3df8f2507-kube-api-access-vntld\") pod \"cert-manager-55b4d954d5-wqmg9\" (UID: \"3ba6351b-0b73-46b6-8d25-b5a3df8f2507\") " pod="cert-manager/cert-manager-55b4d954d5-wqmg9" Jul 22 16:13:10.634843 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.634722 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wqmg9" Jul 22 16:13:10.796538 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:10.796497 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wqmg9"] Jul 22 16:13:10.799710 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:13:10.799680 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ba6351b_0b73_46b6_8d25_b5a3df8f2507.slice/crio-45bc5f1365d2e5ecfb185d0fd1f5178d4033ec217718d15c0f9797f2220acc9f WatchSource:0}: Error finding container 45bc5f1365d2e5ecfb185d0fd1f5178d4033ec217718d15c0f9797f2220acc9f: Status 404 returned error can't find the container with id 45bc5f1365d2e5ecfb185d0fd1f5178d4033ec217718d15c0f9797f2220acc9f Jul 22 16:13:11.625224 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:11.625180 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wqmg9" event={"ID":"3ba6351b-0b73-46b6-8d25-b5a3df8f2507","Type":"ContainerStarted","Data":"f9478cdd43bcf8b424d415d2ccd899c83dc0ed5665cae4338e30abd5f9e73ec8"} Jul 22 16:13:11.625224 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:11.625230 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wqmg9" event={"ID":"3ba6351b-0b73-46b6-8d25-b5a3df8f2507","Type":"ContainerStarted","Data":"45bc5f1365d2e5ecfb185d0fd1f5178d4033ec217718d15c0f9797f2220acc9f"} Jul 22 16:13:21.742883 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.742822 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wqmg9" podStartSLOduration=11.742804913 podStartE2EDuration="11.742804913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:11.683225307 +0000 UTC m=+355.078767143" watchObservedRunningTime="2026-07-22 16:13:21.742804913 +0000 UTC m=+365.138346750" Jul 22 16:13:21.743544 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.743523 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc"] Jul 22 16:13:21.747530 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.747513 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:21.754614 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.754596 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 16:13:21.754873 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.754858 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wgdqm\"" Jul 22 16:13:21.754914 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.754860 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 16:13:21.772375 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.772335 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc"] Jul 22 16:13:21.880853 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.880815 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:21.881032 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.880885 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:21.881032 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.880922 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l462p\" (UniqueName: \"kubernetes.io/projected/ca0423f5-1a3c-4775-986b-2b47823b0737-kube-api-access-l462p\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:21.981504 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.981460 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:21.981877 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.981543 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l462p\" (UniqueName: \"kubernetes.io/projected/ca0423f5-1a3c-4775-986b-2b47823b0737-kube-api-access-l462p\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:21.981877 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.981680 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:21.981877 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.981774 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:21.982067 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:21.982020 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:22.008023 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:22.007958 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l462p\" (UniqueName: \"kubernetes.io/projected/ca0423f5-1a3c-4775-986b-2b47823b0737-kube-api-access-l462p\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:22.056904 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:22.056874 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:22.228379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:22.228354 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc"] Jul 22 16:13:22.230224 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:13:22.230199 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca0423f5_1a3c_4775_986b_2b47823b0737.slice/crio-3a07a9ecd1c8fdd85d70bb0546802fe1514a04433357056dae563e438058c466 WatchSource:0}: Error finding container 3a07a9ecd1c8fdd85d70bb0546802fe1514a04433357056dae563e438058c466: Status 404 returned error can't find the container with id 3a07a9ecd1c8fdd85d70bb0546802fe1514a04433357056dae563e438058c466 Jul 22 16:13:22.669353 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:22.669251 2564 generic.go:356] "Generic (PLEG): container finished" podID="ca0423f5-1a3c-4775-986b-2b47823b0737" containerID="0e9f7019571884c9514e294ca6216a9803e5ea6d78a69c7be77686673894c564" exitCode=0 Jul 22 16:13:22.669353 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:22.669342 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" event={"ID":"ca0423f5-1a3c-4775-986b-2b47823b0737","Type":"ContainerDied","Data":"0e9f7019571884c9514e294ca6216a9803e5ea6d78a69c7be77686673894c564"} Jul 22 16:13:22.669587 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:22.669388 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" event={"ID":"ca0423f5-1a3c-4775-986b-2b47823b0737","Type":"ContainerStarted","Data":"3a07a9ecd1c8fdd85d70bb0546802fe1514a04433357056dae563e438058c466"} Jul 22 16:13:23.674803 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:23.674769 2564 generic.go:356] "Generic (PLEG): container finished" podID="ca0423f5-1a3c-4775-986b-2b47823b0737" containerID="94002ff5759891cd292fb7e032fc02daae8e353a0b5cb8713cc417eafcbaeb8c" exitCode=0 Jul 22 16:13:23.675206 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:23.674856 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" event={"ID":"ca0423f5-1a3c-4775-986b-2b47823b0737","Type":"ContainerDied","Data":"94002ff5759891cd292fb7e032fc02daae8e353a0b5cb8713cc417eafcbaeb8c"} Jul 22 16:13:24.681557 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:24.681525 2564 generic.go:356] "Generic (PLEG): container finished" podID="ca0423f5-1a3c-4775-986b-2b47823b0737" containerID="e35b6aa31a120b62af97b10983fbd21d2b80c04bd6a8249e09ae8cc4245c1e30" exitCode=0 Jul 22 16:13:24.681984 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:24.681610 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" event={"ID":"ca0423f5-1a3c-4775-986b-2b47823b0737","Type":"ContainerDied","Data":"e35b6aa31a120b62af97b10983fbd21d2b80c04bd6a8249e09ae8cc4245c1e30"} Jul 22 16:13:25.818661 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:25.818617 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:25.915735 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:25.915691 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-util\") pod \"ca0423f5-1a3c-4775-986b-2b47823b0737\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " Jul 22 16:13:25.915951 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:25.915751 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-bundle\") pod \"ca0423f5-1a3c-4775-986b-2b47823b0737\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " Jul 22 16:13:25.915951 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:25.915828 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l462p\" (UniqueName: \"kubernetes.io/projected/ca0423f5-1a3c-4775-986b-2b47823b0737-kube-api-access-l462p\") pod \"ca0423f5-1a3c-4775-986b-2b47823b0737\" (UID: \"ca0423f5-1a3c-4775-986b-2b47823b0737\") " Jul 22 16:13:25.916484 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:25.916456 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-bundle" (OuterVolumeSpecName: "bundle") pod "ca0423f5-1a3c-4775-986b-2b47823b0737" (UID: "ca0423f5-1a3c-4775-986b-2b47823b0737"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:13:25.918011 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:25.917976 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca0423f5-1a3c-4775-986b-2b47823b0737-kube-api-access-l462p" (OuterVolumeSpecName: "kube-api-access-l462p") pod "ca0423f5-1a3c-4775-986b-2b47823b0737" (UID: "ca0423f5-1a3c-4775-986b-2b47823b0737"). InnerVolumeSpecName "kube-api-access-l462p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:13:25.921521 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:25.921478 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-util" (OuterVolumeSpecName: "util") pod "ca0423f5-1a3c-4775-986b-2b47823b0737" (UID: "ca0423f5-1a3c-4775-986b-2b47823b0737"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:13:26.016484 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:26.016447 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-bundle\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:13:26.016484 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:26.016482 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l462p\" (UniqueName: \"kubernetes.io/projected/ca0423f5-1a3c-4775-986b-2b47823b0737-kube-api-access-l462p\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:13:26.016484 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:26.016491 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca0423f5-1a3c-4775-986b-2b47823b0737-util\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:13:26.691363 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:26.691320 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" event={"ID":"ca0423f5-1a3c-4775-986b-2b47823b0737","Type":"ContainerDied","Data":"3a07a9ecd1c8fdd85d70bb0546802fe1514a04433357056dae563e438058c466"} Jul 22 16:13:26.691544 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:26.691373 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afsllwc" Jul 22 16:13:26.691544 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:26.691385 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a07a9ecd1c8fdd85d70bb0546802fe1514a04433357056dae563e438058c466" Jul 22 16:13:34.897353 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.897308 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv"] Jul 22 16:13:34.897902 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.897768 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca0423f5-1a3c-4775-986b-2b47823b0737" containerName="extract" Jul 22 16:13:34.897902 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.897787 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca0423f5-1a3c-4775-986b-2b47823b0737" containerName="extract" Jul 22 16:13:34.897902 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.897804 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca0423f5-1a3c-4775-986b-2b47823b0737" containerName="util" Jul 22 16:13:34.897902 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.897814 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca0423f5-1a3c-4775-986b-2b47823b0737" containerName="util" Jul 22 16:13:34.897902 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.897845 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca0423f5-1a3c-4775-986b-2b47823b0737" containerName="pull" Jul 22 16:13:34.897902 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.897853 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca0423f5-1a3c-4775-986b-2b47823b0737" containerName="pull" Jul 22 16:13:34.898194 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.897921 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca0423f5-1a3c-4775-986b-2b47823b0737" containerName="extract" Jul 22 16:13:34.900243 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.900220 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:34.945054 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.945014 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 16:13:34.945268 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.945253 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 16:13:34.945380 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.945363 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x4fjv\"" Jul 22 16:13:34.967019 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.966984 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 16:13:34.968733 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.968709 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 16:13:34.992486 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.992450 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/991db946-f941-4a34-94f4-3b72eca34a92-manifests\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:34.992486 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.992490 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jczpf\" (UniqueName: \"kubernetes.io/projected/991db946-f941-4a34-94f4-3b72eca34a92-kube-api-access-jczpf\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:34.992710 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.992514 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/991db946-f941-4a34-94f4-3b72eca34a92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:34.992710 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.992609 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/991db946-f941-4a34-94f4-3b72eca34a92-webhook-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:34.992710 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:34.992704 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/991db946-f941-4a34-94f4-3b72eca34a92-tmp\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.011296 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.011261 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv"] Jul 22 16:13:35.093167 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.093125 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/991db946-f941-4a34-94f4-3b72eca34a92-webhook-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.093355 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.093185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/991db946-f941-4a34-94f4-3b72eca34a92-tmp\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.093355 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.093215 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/991db946-f941-4a34-94f4-3b72eca34a92-manifests\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.093355 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.093237 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jczpf\" (UniqueName: \"kubernetes.io/projected/991db946-f941-4a34-94f4-3b72eca34a92-kube-api-access-jczpf\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.093355 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.093260 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/991db946-f941-4a34-94f4-3b72eca34a92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.093744 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.093723 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/991db946-f941-4a34-94f4-3b72eca34a92-manifests\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.095565 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.095533 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/991db946-f941-4a34-94f4-3b72eca34a92-tmp\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.095805 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.095786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/991db946-f941-4a34-94f4-3b72eca34a92-apiservice-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.095878 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.095863 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/991db946-f941-4a34-94f4-3b72eca34a92-webhook-cert\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.136874 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.136831 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jczpf\" (UniqueName: \"kubernetes.io/projected/991db946-f941-4a34-94f4-3b72eca34a92-kube-api-access-jczpf\") pod \"opendatahub-operator-controller-manager-587bf8b54-5bgrv\" (UID: \"991db946-f941-4a34-94f4-3b72eca34a92\") " pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.210262 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.210152 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:35.386388 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.386352 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv"] Jul 22 16:13:35.390471 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:13:35.390443 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991db946_f941_4a34_94f4_3b72eca34a92.slice/crio-c140f47ad5d1bed6b9cfef3d3e7163d813c9011ae24b6450c6ab551956732962 WatchSource:0}: Error finding container c140f47ad5d1bed6b9cfef3d3e7163d813c9011ae24b6450c6ab551956732962: Status 404 returned error can't find the container with id c140f47ad5d1bed6b9cfef3d3e7163d813c9011ae24b6450c6ab551956732962 Jul 22 16:13:35.731583 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:35.731543 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" event={"ID":"991db946-f941-4a34-94f4-3b72eca34a92","Type":"ContainerStarted","Data":"c140f47ad5d1bed6b9cfef3d3e7163d813c9011ae24b6450c6ab551956732962"} Jul 22 16:13:38.747408 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:38.747372 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" event={"ID":"991db946-f941-4a34-94f4-3b72eca34a92","Type":"ContainerStarted","Data":"056a51ce50f401e375fefaf2ed1620b09b775ae12af7796763f9941f19b0b7b2"} Jul 22 16:13:39.752977 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:39.752941 2564 generic.go:356] "Generic (PLEG): container finished" podID="991db946-f941-4a34-94f4-3b72eca34a92" containerID="056a51ce50f401e375fefaf2ed1620b09b775ae12af7796763f9941f19b0b7b2" exitCode=0 Jul 22 16:13:39.753475 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:39.753036 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" event={"ID":"991db946-f941-4a34-94f4-3b72eca34a92","Type":"ContainerDied","Data":"056a51ce50f401e375fefaf2ed1620b09b775ae12af7796763f9941f19b0b7b2"} Jul 22 16:13:40.765011 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:40.764973 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" event={"ID":"991db946-f941-4a34-94f4-3b72eca34a92","Type":"ContainerStarted","Data":"4b42a0e96a298de331f719000d720281023df720d58b2616846b5b08b6075ac1"} Jul 22 16:13:40.765402 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:40.765071 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:40.819413 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:40.819352 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" podStartSLOduration=2.063680689 podStartE2EDuration="6.819335432s" totalImagesPullingTime="3.747815601s" totalInitContainerRuntime="1.007839142s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:40.819127757 +0000 UTC m=+384.214669591" watchObservedRunningTime="2026-07-22 16:13:40.819335432 +0000 UTC m=+384.214877266" Jul 22 16:13:51.771662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:51.771609 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-587bf8b54-5bgrv" Jul 22 16:13:56.499991 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.499949 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj"] Jul 22 16:13:56.508479 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.508452 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.514267 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.514229 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 22 16:13:56.514444 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.514238 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 22 16:13:56.514444 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.514298 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 16:13:56.516292 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.515916 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 16:13:56.550584 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.550542 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj"] Jul 22 16:13:56.577425 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.577392 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/479e54eb-ec53-41a5-8b4c-2d7a5363bba0-tmp\") pod \"kube-auth-proxy-9b87bf6fb-7rljj\" (UID: \"479e54eb-ec53-41a5-8b4c-2d7a5363bba0\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.577654 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.577446 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/479e54eb-ec53-41a5-8b4c-2d7a5363bba0-tls-certs\") pod \"kube-auth-proxy-9b87bf6fb-7rljj\" (UID: \"479e54eb-ec53-41a5-8b4c-2d7a5363bba0\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.577654 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.577544 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crtvs\" (UniqueName: \"kubernetes.io/projected/479e54eb-ec53-41a5-8b4c-2d7a5363bba0-kube-api-access-crtvs\") pod \"kube-auth-proxy-9b87bf6fb-7rljj\" (UID: \"479e54eb-ec53-41a5-8b4c-2d7a5363bba0\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.678730 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.678606 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crtvs\" (UniqueName: \"kubernetes.io/projected/479e54eb-ec53-41a5-8b4c-2d7a5363bba0-kube-api-access-crtvs\") pod \"kube-auth-proxy-9b87bf6fb-7rljj\" (UID: \"479e54eb-ec53-41a5-8b4c-2d7a5363bba0\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.678913 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.678731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/479e54eb-ec53-41a5-8b4c-2d7a5363bba0-tmp\") pod \"kube-auth-proxy-9b87bf6fb-7rljj\" (UID: \"479e54eb-ec53-41a5-8b4c-2d7a5363bba0\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.678913 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.678774 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/479e54eb-ec53-41a5-8b4c-2d7a5363bba0-tls-certs\") pod \"kube-auth-proxy-9b87bf6fb-7rljj\" (UID: \"479e54eb-ec53-41a5-8b4c-2d7a5363bba0\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.681459 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.681376 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/479e54eb-ec53-41a5-8b4c-2d7a5363bba0-tmp\") pod \"kube-auth-proxy-9b87bf6fb-7rljj\" (UID: \"479e54eb-ec53-41a5-8b4c-2d7a5363bba0\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.681860 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.681836 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/479e54eb-ec53-41a5-8b4c-2d7a5363bba0-tls-certs\") pod \"kube-auth-proxy-9b87bf6fb-7rljj\" (UID: \"479e54eb-ec53-41a5-8b4c-2d7a5363bba0\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.710786 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.710741 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7j6jk"] Jul 22 16:13:56.714524 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.714421 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:13:56.716741 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.716707 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crtvs\" (UniqueName: \"kubernetes.io/projected/479e54eb-ec53-41a5-8b4c-2d7a5363bba0-kube-api-access-crtvs\") pod \"kube-auth-proxy-9b87bf6fb-7rljj\" (UID: \"479e54eb-ec53-41a5-8b4c-2d7a5363bba0\") " pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.727828 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.727793 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 22 16:13:56.729519 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.729494 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cz652\"" Jul 22 16:13:56.740612 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.740575 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7j6jk"] Jul 22 16:13:56.779807 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.779770 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtvnn\" (UniqueName: \"kubernetes.io/projected/bb0554b7-069b-4ee9-bd75-500d7283e11c-kube-api-access-jtvnn\") pod \"odh-model-controller-7998c67dc4-7j6jk\" (UID: \"bb0554b7-069b-4ee9-bd75-500d7283e11c\") " pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:13:56.780006 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.779834 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb0554b7-069b-4ee9-bd75-500d7283e11c-cert\") pod \"odh-model-controller-7998c67dc4-7j6jk\" (UID: \"bb0554b7-069b-4ee9-bd75-500d7283e11c\") " pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:13:56.819755 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.819722 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" Jul 22 16:13:56.881260 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.881205 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtvnn\" (UniqueName: \"kubernetes.io/projected/bb0554b7-069b-4ee9-bd75-500d7283e11c-kube-api-access-jtvnn\") pod \"odh-model-controller-7998c67dc4-7j6jk\" (UID: \"bb0554b7-069b-4ee9-bd75-500d7283e11c\") " pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:13:56.881432 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.881283 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb0554b7-069b-4ee9-bd75-500d7283e11c-cert\") pod \"odh-model-controller-7998c67dc4-7j6jk\" (UID: \"bb0554b7-069b-4ee9-bd75-500d7283e11c\") " pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:13:56.881432 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:13:56.881413 2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 22 16:13:56.881541 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:13:56.881470 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb0554b7-069b-4ee9-bd75-500d7283e11c-cert podName:bb0554b7-069b-4ee9-bd75-500d7283e11c nodeName:}" failed. No retries permitted until 2026-07-22 16:13:57.381454541 +0000 UTC m=+400.776996353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bb0554b7-069b-4ee9-bd75-500d7283e11c-cert") pod "odh-model-controller-7998c67dc4-7j6jk" (UID: "bb0554b7-069b-4ee9-bd75-500d7283e11c") : secret "odh-model-controller-webhook-cert" not found Jul 22 16:13:56.962331 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:56.962256 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtvnn\" (UniqueName: \"kubernetes.io/projected/bb0554b7-069b-4ee9-bd75-500d7283e11c-kube-api-access-jtvnn\") pod \"odh-model-controller-7998c67dc4-7j6jk\" (UID: \"bb0554b7-069b-4ee9-bd75-500d7283e11c\") " pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:13:57.046097 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:57.046061 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj"] Jul 22 16:13:57.049932 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:13:57.049895 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod479e54eb_ec53_41a5_8b4c_2d7a5363bba0.slice/crio-1dee46b8cfdc644bd40fb6262c055dc80bc43360a7e864ac49fce4dd46ef9301 WatchSource:0}: Error finding container 1dee46b8cfdc644bd40fb6262c055dc80bc43360a7e864ac49fce4dd46ef9301: Status 404 returned error can't find the container with id 1dee46b8cfdc644bd40fb6262c055dc80bc43360a7e864ac49fce4dd46ef9301 Jul 22 16:13:57.386423 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:57.386369 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb0554b7-069b-4ee9-bd75-500d7283e11c-cert\") pod \"odh-model-controller-7998c67dc4-7j6jk\" (UID: \"bb0554b7-069b-4ee9-bd75-500d7283e11c\") " pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:13:57.389003 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:57.388974 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb0554b7-069b-4ee9-bd75-500d7283e11c-cert\") pod \"odh-model-controller-7998c67dc4-7j6jk\" (UID: \"bb0554b7-069b-4ee9-bd75-500d7283e11c\") " pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:13:57.632368 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:57.632324 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:13:57.826228 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:57.826176 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7j6jk"] Jul 22 16:13:57.835135 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:13:57.835095 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0554b7_069b_4ee9_bd75_500d7283e11c.slice/crio-6b63365419d0a486be289051696c0dc0f935d01a8dfaba11e74584d89450efc7 WatchSource:0}: Error finding container 6b63365419d0a486be289051696c0dc0f935d01a8dfaba11e74584d89450efc7: Status 404 returned error can't find the container with id 6b63365419d0a486be289051696c0dc0f935d01a8dfaba11e74584d89450efc7 Jul 22 16:13:57.849342 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:57.849276 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" event={"ID":"bb0554b7-069b-4ee9-bd75-500d7283e11c","Type":"ContainerStarted","Data":"6b63365419d0a486be289051696c0dc0f935d01a8dfaba11e74584d89450efc7"} Jul 22 16:13:57.850817 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:57.850773 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" event={"ID":"479e54eb-ec53-41a5-8b4c-2d7a5363bba0","Type":"ContainerStarted","Data":"1dee46b8cfdc644bd40fb6262c055dc80bc43360a7e864ac49fce4dd46ef9301"} Jul 22 16:13:59.179161 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:59.179119 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hmgdw"] Jul 22 16:13:59.191385 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:59.190970 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:13:59.196399 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:13:59.196183 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"model-serving-api-54d89bd5fc-hmgdw\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="d14267c3-dc35-40e5-8fc0-6d6a2abd03e3" pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:13:59.197860 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:13:59.197814 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"model-serving-api-tls\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" type="*v1.Secret" Jul 22 16:13:59.203605 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:59.203509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d14267c3-dc35-40e5-8fc0-6d6a2abd03e3-tls-certs\") pod \"model-serving-api-54d89bd5fc-hmgdw\" (UID: \"d14267c3-dc35-40e5-8fc0-6d6a2abd03e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:13:59.203605 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:59.203569 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvs9c\" (UniqueName: \"kubernetes.io/projected/d14267c3-dc35-40e5-8fc0-6d6a2abd03e3-kube-api-access-cvs9c\") pod \"model-serving-api-54d89bd5fc-hmgdw\" (UID: \"d14267c3-dc35-40e5-8fc0-6d6a2abd03e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:13:59.219519 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:13:59.219479 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"model-serving-api-dockercfg-bb2b9\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-bb2b9\"" type="*v1.Secret" Jul 22 16:13:59.276406 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:59.276369 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hmgdw"] Jul 22 16:13:59.305117 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:59.305059 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d14267c3-dc35-40e5-8fc0-6d6a2abd03e3-tls-certs\") pod \"model-serving-api-54d89bd5fc-hmgdw\" (UID: \"d14267c3-dc35-40e5-8fc0-6d6a2abd03e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:13:59.305311 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:59.305128 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvs9c\" (UniqueName: \"kubernetes.io/projected/d14267c3-dc35-40e5-8fc0-6d6a2abd03e3-kube-api-access-cvs9c\") pod \"model-serving-api-54d89bd5fc-hmgdw\" (UID: \"d14267c3-dc35-40e5-8fc0-6d6a2abd03e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:13:59.356762 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:13:59.356727 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvs9c\" (UniqueName: \"kubernetes.io/projected/d14267c3-dc35-40e5-8fc0-6d6a2abd03e3-kube-api-access-cvs9c\") pod \"model-serving-api-54d89bd5fc-hmgdw\" (UID: \"d14267c3-dc35-40e5-8fc0-6d6a2abd03e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:14:00.306498 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:14:00.306449 2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: failed to sync secret cache: timed out waiting for the condition Jul 22 16:14:00.307007 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:14:00.306564 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d14267c3-dc35-40e5-8fc0-6d6a2abd03e3-tls-certs podName:d14267c3-dc35-40e5-8fc0-6d6a2abd03e3 nodeName:}" failed. No retries permitted until 2026-07-22 16:14:00.806540078 +0000 UTC m=+404.202081895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d14267c3-dc35-40e5-8fc0-6d6a2abd03e3-tls-certs") pod "model-serving-api-54d89bd5fc-hmgdw" (UID: "d14267c3-dc35-40e5-8fc0-6d6a2abd03e3") : failed to sync secret cache: timed out waiting for the condition Jul 22 16:14:00.433113 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.433064 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t"] Jul 22 16:14:00.442265 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.442237 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.460808 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.460775 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 16:14:00.460957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.460859 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zxf5w\"" Jul 22 16:14:00.461344 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.461329 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 16:14:00.462254 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.462237 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 16:14:00.463839 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.463819 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 16:14:00.495159 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.495125 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t"] Jul 22 16:14:00.517871 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.517833 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.517871 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.517868 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdq2p\" (UniqueName: \"kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-kube-api-access-wdq2p\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.518108 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.517892 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.518108 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.518035 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-cacerts\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.518108 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.518069 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.518108 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.518098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/08dae35b-9903-4b3e-88da-ad2d593426a9-local-certs\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.518281 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.518126 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-token\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.619282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.619201 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.619282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.619245 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdq2p\" (UniqueName: \"kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-kube-api-access-wdq2p\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.619282 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.619264 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.619599 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.619303 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-cacerts\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.619599 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.619337 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.619599 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.619374 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/08dae35b-9903-4b3e-88da-ad2d593426a9-local-certs\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.619599 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.619403 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-token\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.619987 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.619931 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.622412 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.622356 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/08dae35b-9903-4b3e-88da-ad2d593426a9-local-certs\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.622546 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.622511 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-cacerts\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.622831 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.622800 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.623231 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.623198 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.640772 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.640728 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-token\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.642439 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.642358 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdq2p\" (UniqueName: \"kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-kube-api-access-wdq2p\") pod \"istiod-openshift-gateway-7574459656-qtj6t\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.732452 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.732357 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-bb2b9\"" Jul 22 16:14:00.753787 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.753754 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:00.799438 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.799405 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 22 16:14:00.821446 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.821404 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d14267c3-dc35-40e5-8fc0-6d6a2abd03e3-tls-certs\") pod \"model-serving-api-54d89bd5fc-hmgdw\" (UID: \"d14267c3-dc35-40e5-8fc0-6d6a2abd03e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:14:00.825529 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:00.825497 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d14267c3-dc35-40e5-8fc0-6d6a2abd03e3-tls-certs\") pod \"model-serving-api-54d89bd5fc-hmgdw\" (UID: \"d14267c3-dc35-40e5-8fc0-6d6a2abd03e3\") " pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:14:01.005983 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:01.005945 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:14:01.586564 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:01.586525 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t"] Jul 22 16:14:01.590603 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:14:01.590572 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08dae35b_9903_4b3e_88da_ad2d593426a9.slice/crio-854619731ea54c4f0aad7628443e9927ce019c1cbe8d8ad163b4f1cdfa9e5d28 WatchSource:0}: Error finding container 854619731ea54c4f0aad7628443e9927ce019c1cbe8d8ad163b4f1cdfa9e5d28: Status 404 returned error can't find the container with id 854619731ea54c4f0aad7628443e9927ce019c1cbe8d8ad163b4f1cdfa9e5d28 Jul 22 16:14:01.632673 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:01.632645 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hmgdw"] Jul 22 16:14:01.635393 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:14:01.635363 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14267c3_dc35_40e5_8fc0_6d6a2abd03e3.slice/crio-1289cdf52f01a8151bd5fb19020a74a15e165f96ccd16adc36b97f75c6474424 WatchSource:0}: Error finding container 1289cdf52f01a8151bd5fb19020a74a15e165f96ccd16adc36b97f75c6474424: Status 404 returned error can't find the container with id 1289cdf52f01a8151bd5fb19020a74a15e165f96ccd16adc36b97f75c6474424 Jul 22 16:14:01.875950 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:01.875868 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" event={"ID":"bb0554b7-069b-4ee9-bd75-500d7283e11c","Type":"ContainerStarted","Data":"1fdf8a3c9c84e392e2f946571b81c9af0e0d9726ab0448f0a95fb6b903ccc0f4"} Jul 22 16:14:01.876305 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:01.875993 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:14:01.878074 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:01.878042 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" event={"ID":"479e54eb-ec53-41a5-8b4c-2d7a5363bba0","Type":"ContainerStarted","Data":"c8828d8a0a9d58bc40c726dbde90e2268889155643e6a88637fa79f4230e66a6"} Jul 22 16:14:01.879787 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:01.879753 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" event={"ID":"d14267c3-dc35-40e5-8fc0-6d6a2abd03e3","Type":"ContainerStarted","Data":"1289cdf52f01a8151bd5fb19020a74a15e165f96ccd16adc36b97f75c6474424"} Jul 22 16:14:01.881222 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:01.881193 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" event={"ID":"08dae35b-9903-4b3e-88da-ad2d593426a9","Type":"ContainerStarted","Data":"854619731ea54c4f0aad7628443e9927ce019c1cbe8d8ad163b4f1cdfa9e5d28"} Jul 22 16:14:01.918316 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:01.918202 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" podStartSLOduration=2.320055131 podStartE2EDuration="5.9181861s" totalImagesPullingTime="3.598130969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:14:01.916412156 +0000 UTC m=+405.311954001" watchObservedRunningTime="2026-07-22 16:14:01.9181861 +0000 UTC m=+405.313727936" Jul 22 16:14:01.963614 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:01.963555 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9b87bf6fb-7rljj" podStartSLOduration=1.639204941 podStartE2EDuration="5.96353635s" totalImagesPullingTime="4.324331409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:14:01.960842331 +0000 UTC m=+405.356384166" watchObservedRunningTime="2026-07-22 16:14:01.96353635 +0000 UTC m=+405.359078226" Jul 22 16:14:04.715874 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:04.715826 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 22 16:14:04.716149 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:04.715928 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 22 16:14:04.900728 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:04.900687 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" event={"ID":"d14267c3-dc35-40e5-8fc0-6d6a2abd03e3","Type":"ContainerStarted","Data":"a39e9d5285f5cff500d11b41178808126f9ad7657e05f53acd04846c4ebaca38"} Jul 22 16:14:04.900963 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:04.900844 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:14:04.902661 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:04.902600 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" event={"ID":"08dae35b-9903-4b3e-88da-ad2d593426a9","Type":"ContainerStarted","Data":"149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab"} Jul 22 16:14:04.902819 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:04.902690 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:05.131520 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:05.131462 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" podStartSLOduration=2.009088395 podStartE2EDuration="5.131445376s" totalImagesPullingTime="3.122356981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:14:05.131070574 +0000 UTC m=+408.526612409" watchObservedRunningTime="2026-07-22 16:14:05.131445376 +0000 UTC m=+408.526987210" Jul 22 16:14:05.132491 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:05.132448 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" podStartSLOduration=3.054011051 podStartE2EDuration="6.132439236s" totalImagesPullingTime="3.078428185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:14:04.949580905 +0000 UTC m=+408.345122740" watchObservedRunningTime="2026-07-22 16:14:05.132439236 +0000 UTC m=+408.527981067" Jul 22 16:14:05.909023 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:05.908990 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-qtj6t container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 16:14:05.909413 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:05.909041 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" podUID="08dae35b-9903-4b3e-88da-ad2d593426a9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 16:14:08.908834 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:08.908800 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:12.889178 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:12.889150 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7j6jk" Jul 22 16:14:13.973816 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:13.973770 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s"] Jul 22 16:14:13.981903 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:13.981877 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" Jul 22 16:14:13.987773 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:13.987743 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 22 16:14:13.988051 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:13.988033 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-88qz2\"" Jul 22 16:14:14.045580 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.045541 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzbfg\" (UniqueName: \"kubernetes.io/projected/e1ec9110-4bac-4048-bf96-059f1a209734-kube-api-access-pzbfg\") pod \"kserve-controller-manager-d6f4b65b8-lmx5s\" (UID: \"e1ec9110-4bac-4048-bf96-059f1a209734\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" Jul 22 16:14:14.045769 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.045696 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1ec9110-4bac-4048-bf96-059f1a209734-cert\") pod \"kserve-controller-manager-d6f4b65b8-lmx5s\" (UID: \"e1ec9110-4bac-4048-bf96-059f1a209734\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" Jul 22 16:14:14.071313 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.071276 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz"] Jul 22 16:14:14.073756 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.073736 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" Jul 22 16:14:14.085019 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.084995 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6lwgp\"" Jul 22 16:14:14.091224 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.091203 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 22 16:14:14.108447 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.108413 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s"] Jul 22 16:14:14.146905 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.146867 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1ec9110-4bac-4048-bf96-059f1a209734-cert\") pod \"kserve-controller-manager-d6f4b65b8-lmx5s\" (UID: \"e1ec9110-4bac-4048-bf96-059f1a209734\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" Jul 22 16:14:14.147085 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.146912 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ksh4\" (UniqueName: \"kubernetes.io/projected/5f36a9fc-12a1-4d97-baea-9255f80f5601-kube-api-access-8ksh4\") pod \"llmisvc-controller-manager-777c6bd7d5-zngzz\" (UID: \"5f36a9fc-12a1-4d97-baea-9255f80f5601\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" Jul 22 16:14:14.147085 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.146937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzbfg\" (UniqueName: \"kubernetes.io/projected/e1ec9110-4bac-4048-bf96-059f1a209734-kube-api-access-pzbfg\") pod \"kserve-controller-manager-d6f4b65b8-lmx5s\" (UID: \"e1ec9110-4bac-4048-bf96-059f1a209734\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" Jul 22 16:14:14.147085 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.146953 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f36a9fc-12a1-4d97-baea-9255f80f5601-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zngzz\" (UID: \"5f36a9fc-12a1-4d97-baea-9255f80f5601\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" Jul 22 16:14:14.149923 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.149893 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1ec9110-4bac-4048-bf96-059f1a209734-cert\") pod \"kserve-controller-manager-d6f4b65b8-lmx5s\" (UID: \"e1ec9110-4bac-4048-bf96-059f1a209734\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" Jul 22 16:14:14.178287 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.178253 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzbfg\" (UniqueName: \"kubernetes.io/projected/e1ec9110-4bac-4048-bf96-059f1a209734-kube-api-access-pzbfg\") pod \"kserve-controller-manager-d6f4b65b8-lmx5s\" (UID: \"e1ec9110-4bac-4048-bf96-059f1a209734\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" Jul 22 16:14:14.228589 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.228510 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz"] Jul 22 16:14:14.247772 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.247740 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ksh4\" (UniqueName: \"kubernetes.io/projected/5f36a9fc-12a1-4d97-baea-9255f80f5601-kube-api-access-8ksh4\") pod \"llmisvc-controller-manager-777c6bd7d5-zngzz\" (UID: \"5f36a9fc-12a1-4d97-baea-9255f80f5601\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" Jul 22 16:14:14.247952 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.247777 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f36a9fc-12a1-4d97-baea-9255f80f5601-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zngzz\" (UID: \"5f36a9fc-12a1-4d97-baea-9255f80f5601\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" Jul 22 16:14:14.250191 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.250167 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f36a9fc-12a1-4d97-baea-9255f80f5601-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zngzz\" (UID: \"5f36a9fc-12a1-4d97-baea-9255f80f5601\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" Jul 22 16:14:14.263293 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.263266 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ksh4\" (UniqueName: \"kubernetes.io/projected/5f36a9fc-12a1-4d97-baea-9255f80f5601-kube-api-access-8ksh4\") pod \"llmisvc-controller-manager-777c6bd7d5-zngzz\" (UID: \"5f36a9fc-12a1-4d97-baea-9255f80f5601\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" Jul 22 16:14:14.292340 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.292303 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" Jul 22 16:14:14.384142 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.384099 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" Jul 22 16:14:14.460342 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.460315 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s"] Jul 22 16:14:14.460879 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:14:14.460851 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ec9110_4bac_4048_bf96_059f1a209734.slice/crio-ae202ab75e08308caec57b8939cc1cad5a26a579425b9eda4ee84d46b5988900 WatchSource:0}: Error finding container ae202ab75e08308caec57b8939cc1cad5a26a579425b9eda4ee84d46b5988900: Status 404 returned error can't find the container with id ae202ab75e08308caec57b8939cc1cad5a26a579425b9eda4ee84d46b5988900 Jul 22 16:14:14.541464 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.541433 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz"] Jul 22 16:14:14.542511 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:14:14.542471 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5f36a9fc_12a1_4d97_baea_9255f80f5601.slice/crio-644bc349e0676e3eae4b0e0cf3da6a940fe6fc2db76cc0d857f202e37d7d5450 WatchSource:0}: Error finding container 644bc349e0676e3eae4b0e0cf3da6a940fe6fc2db76cc0d857f202e37d7d5450: Status 404 returned error can't find the container with id 644bc349e0676e3eae4b0e0cf3da6a940fe6fc2db76cc0d857f202e37d7d5450 Jul 22 16:14:14.953335 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.953241 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" event={"ID":"5f36a9fc-12a1-4d97-baea-9255f80f5601","Type":"ContainerStarted","Data":"644bc349e0676e3eae4b0e0cf3da6a940fe6fc2db76cc0d857f202e37d7d5450"} Jul 22 16:14:14.954504 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:14.954464 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" event={"ID":"e1ec9110-4bac-4048-bf96-059f1a209734","Type":"ContainerStarted","Data":"ae202ab75e08308caec57b8939cc1cad5a26a579425b9eda4ee84d46b5988900"} Jul 22 16:14:15.911899 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:15.911869 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hmgdw" Jul 22 16:14:16.965233 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:16.965192 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" event={"ID":"e1ec9110-4bac-4048-bf96-059f1a209734","Type":"ContainerStarted","Data":"bead0cb98fde2eb658317fe193483f781a0876429cc4c14fd2aac2b412a24143"} Jul 22 16:14:16.965659 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:16.965260 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" Jul 22 16:14:17.005904 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:17.005848 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" podStartSLOduration=1.678902856 podStartE2EDuration="4.00583368s" totalImagesPullingTime="2.326930824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:14:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:14:17.005015408 +0000 UTC m=+420.400557241" watchObservedRunningTime="2026-07-22 16:14:17.00583368 +0000 UTC m=+420.401375513" Jul 22 16:14:18.977528 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:18.977483 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" event={"ID":"5f36a9fc-12a1-4d97-baea-9255f80f5601","Type":"ContainerStarted","Data":"06911c3dd05e020a7390715d6148fb56d16606d1f67987ce12ecc105eb6c7f75"} Jul 22 16:14:18.977975 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:18.977572 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" Jul 22 16:14:19.001152 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:19.001096 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" podStartSLOduration=2.499497972 podStartE2EDuration="6.001080602s" totalImagesPullingTime="3.50158263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:14:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:14:18.999410812 +0000 UTC m=+422.394952649" watchObservedRunningTime="2026-07-22 16:14:19.001080602 +0000 UTC m=+422.396622437" Jul 22 16:14:19.917141 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:19.917101 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t"] Jul 22 16:14:19.917327 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:19.917307 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" podUID="08dae35b-9903-4b3e-88da-ad2d593426a9" containerName="discovery" containerID="cri-o://149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab" gracePeriod=30 Jul 22 16:14:20.170294 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.170226 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:20.305375 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.305333 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-kubeconfig\") pod \"08dae35b-9903-4b3e-88da-ad2d593426a9\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " Jul 22 16:14:20.305565 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.305408 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-token\") pod \"08dae35b-9903-4b3e-88da-ad2d593426a9\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " Jul 22 16:14:20.305565 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.305429 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/08dae35b-9903-4b3e-88da-ad2d593426a9-local-certs\") pod \"08dae35b-9903-4b3e-88da-ad2d593426a9\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " Jul 22 16:14:20.305565 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.305453 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-ca-configmap\") pod \"08dae35b-9903-4b3e-88da-ad2d593426a9\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " Jul 22 16:14:20.305565 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.305469 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdq2p\" (UniqueName: \"kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-kube-api-access-wdq2p\") pod \"08dae35b-9903-4b3e-88da-ad2d593426a9\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " Jul 22 16:14:20.305565 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.305524 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-dns-cert\") pod \"08dae35b-9903-4b3e-88da-ad2d593426a9\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " Jul 22 16:14:20.305565 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.305560 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-cacerts\") pod \"08dae35b-9903-4b3e-88da-ad2d593426a9\" (UID: \"08dae35b-9903-4b3e-88da-ad2d593426a9\") " Jul 22 16:14:20.306193 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.306166 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "08dae35b-9903-4b3e-88da-ad2d593426a9" (UID: "08dae35b-9903-4b3e-88da-ad2d593426a9"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:14:20.307986 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.307961 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "08dae35b-9903-4b3e-88da-ad2d593426a9" (UID: "08dae35b-9903-4b3e-88da-ad2d593426a9"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:14:20.308181 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.308148 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-cacerts" (OuterVolumeSpecName: "cacerts") pod "08dae35b-9903-4b3e-88da-ad2d593426a9" (UID: "08dae35b-9903-4b3e-88da-ad2d593426a9"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:14:20.308272 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.308196 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-kube-api-access-wdq2p" (OuterVolumeSpecName: "kube-api-access-wdq2p") pod "08dae35b-9903-4b3e-88da-ad2d593426a9" (UID: "08dae35b-9903-4b3e-88da-ad2d593426a9"). InnerVolumeSpecName "kube-api-access-wdq2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:14:20.308272 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.308214 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08dae35b-9903-4b3e-88da-ad2d593426a9-local-certs" (OuterVolumeSpecName: "local-certs") pod "08dae35b-9903-4b3e-88da-ad2d593426a9" (UID: "08dae35b-9903-4b3e-88da-ad2d593426a9"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:14:20.308272 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.308238 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "08dae35b-9903-4b3e-88da-ad2d593426a9" (UID: "08dae35b-9903-4b3e-88da-ad2d593426a9"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:14:20.308369 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.308264 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "08dae35b-9903-4b3e-88da-ad2d593426a9" (UID: "08dae35b-9903-4b3e-88da-ad2d593426a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:14:20.406928 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.406885 2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-cacerts\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:14:20.406928 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.406919 2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-kubeconfig\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:14:20.406928 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.406930 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-token\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:14:20.406928 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.406939 2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/08dae35b-9903-4b3e-88da-ad2d593426a9-local-certs\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:14:20.407190 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.406948 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-ca-configmap\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:14:20.407190 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.406956 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdq2p\" (UniqueName: \"kubernetes.io/projected/08dae35b-9903-4b3e-88da-ad2d593426a9-kube-api-access-wdq2p\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:14:20.407190 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.406964 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/08dae35b-9903-4b3e-88da-ad2d593426a9-istio-csr-dns-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:14:20.988849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.988810 2564 generic.go:356] "Generic (PLEG): container finished" podID="08dae35b-9903-4b3e-88da-ad2d593426a9" containerID="149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab" exitCode=0 Jul 22 16:14:20.989007 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.988874 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" Jul 22 16:14:20.989007 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.988875 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" event={"ID":"08dae35b-9903-4b3e-88da-ad2d593426a9","Type":"ContainerDied","Data":"149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab"} Jul 22 16:14:20.989112 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.989012 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t" event={"ID":"08dae35b-9903-4b3e-88da-ad2d593426a9","Type":"ContainerDied","Data":"854619731ea54c4f0aad7628443e9927ce019c1cbe8d8ad163b4f1cdfa9e5d28"} Jul 22 16:14:20.989112 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.989033 2564 scope.go:117] "RemoveContainer" containerID="149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab" Jul 22 16:14:20.998052 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.998031 2564 scope.go:117] "RemoveContainer" containerID="149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab" Jul 22 16:14:20.998335 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:14:20.998314 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab\": container with ID starting with 149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab not found: ID does not exist" containerID="149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab" Jul 22 16:14:20.998391 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:20.998344 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab"} err="failed to get container status \"149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab\": rpc error: code = NotFound desc = could not find container \"149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab\": container with ID starting with 149f0089431d856104b3a6df7cf2df85920f80e6ad9620ce55d0bc8eb47db1ab not found: ID does not exist" Jul 22 16:14:21.016649 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:21.016607 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t"] Jul 22 16:14:21.019446 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:21.019425 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qtj6t"] Jul 22 16:14:21.128469 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:21.128435 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08dae35b-9903-4b3e-88da-ad2d593426a9" path="/var/lib/kubelet/pods/08dae35b-9903-4b3e-88da-ad2d593426a9/volumes" Jul 22 16:14:47.976787 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:47.976699 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lmx5s" Jul 22 16:14:49.985564 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:49.985533 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zngzz" Jul 22 16:14:57.133010 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.132976 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fgn4b"] Jul 22 16:14:57.133379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.133337 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08dae35b-9903-4b3e-88da-ad2d593426a9" containerName="discovery" Jul 22 16:14:57.133379 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.133349 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="08dae35b-9903-4b3e-88da-ad2d593426a9" containerName="discovery" Jul 22 16:14:57.133448 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.133418 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="08dae35b-9903-4b3e-88da-ad2d593426a9" containerName="discovery" Jul 22 16:14:57.138064 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.138038 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" Jul 22 16:14:57.146757 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.146706 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 16:14:57.146957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.146930 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 16:14:57.150577 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.149229 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-zsh5j\"" Jul 22 16:14:57.157113 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.157079 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fgn4b"] Jul 22 16:14:57.209692 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.209657 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qk2g\" (UniqueName: \"kubernetes.io/projected/8758c958-77f7-4174-b997-3747665753a3-kube-api-access-5qk2g\") pod \"kuadrant-operator-catalog-fgn4b\" (UID: \"8758c958-77f7-4174-b997-3747665753a3\") " pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" Jul 22 16:14:57.310311 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.310274 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qk2g\" (UniqueName: \"kubernetes.io/projected/8758c958-77f7-4174-b997-3747665753a3-kube-api-access-5qk2g\") pod \"kuadrant-operator-catalog-fgn4b\" (UID: \"8758c958-77f7-4174-b997-3747665753a3\") " pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" Jul 22 16:14:57.320513 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.320480 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qk2g\" (UniqueName: \"kubernetes.io/projected/8758c958-77f7-4174-b997-3747665753a3-kube-api-access-5qk2g\") pod \"kuadrant-operator-catalog-fgn4b\" (UID: \"8758c958-77f7-4174-b997-3747665753a3\") " pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" Jul 22 16:14:57.440486 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.440392 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fgn4b"] Jul 22 16:14:57.440715 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.440699 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" Jul 22 16:14:57.610899 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.610871 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fgn4b"] Jul 22 16:14:57.612911 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:14:57.612883 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8758c958_77f7_4174_b997_3747665753a3.slice/crio-96ca81d1cd031aca12dd5cb4ef283aeab1003b65244756c2a801fc771db70370 WatchSource:0}: Error finding container 96ca81d1cd031aca12dd5cb4ef283aeab1003b65244756c2a801fc771db70370: Status 404 returned error can't find the container with id 96ca81d1cd031aca12dd5cb4ef283aeab1003b65244756c2a801fc771db70370 Jul 22 16:14:57.651043 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.651005 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gjrpl"] Jul 22 16:14:57.653920 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.653902 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" Jul 22 16:14:57.666604 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.666570 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gjrpl"] Jul 22 16:14:57.713897 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.713791 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-268md\" (UniqueName: \"kubernetes.io/projected/95bb3861-c7b6-44eb-8976-a2c62964e1ad-kube-api-access-268md\") pod \"kuadrant-operator-catalog-gjrpl\" (UID: \"95bb3861-c7b6-44eb-8976-a2c62964e1ad\") " pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" Jul 22 16:14:57.814723 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.814679 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-268md\" (UniqueName: \"kubernetes.io/projected/95bb3861-c7b6-44eb-8976-a2c62964e1ad-kube-api-access-268md\") pod \"kuadrant-operator-catalog-gjrpl\" (UID: \"95bb3861-c7b6-44eb-8976-a2c62964e1ad\") " pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" Jul 22 16:14:57.827027 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.826993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-268md\" (UniqueName: \"kubernetes.io/projected/95bb3861-c7b6-44eb-8976-a2c62964e1ad-kube-api-access-268md\") pod \"kuadrant-operator-catalog-gjrpl\" (UID: \"95bb3861-c7b6-44eb-8976-a2c62964e1ad\") " pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" Jul 22 16:14:57.964388 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:57.964281 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" Jul 22 16:14:58.108349 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:58.108319 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gjrpl"] Jul 22 16:14:58.110057 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:14:58.110030 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95bb3861_c7b6_44eb_8976_a2c62964e1ad.slice/crio-da25736989a2ff89f41c50bf4fe2f43bbdb1cafc55cc5a23476104428082748b WatchSource:0}: Error finding container da25736989a2ff89f41c50bf4fe2f43bbdb1cafc55cc5a23476104428082748b: Status 404 returned error can't find the container with id da25736989a2ff89f41c50bf4fe2f43bbdb1cafc55cc5a23476104428082748b Jul 22 16:14:58.168573 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:58.168536 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" event={"ID":"95bb3861-c7b6-44eb-8976-a2c62964e1ad","Type":"ContainerStarted","Data":"da25736989a2ff89f41c50bf4fe2f43bbdb1cafc55cc5a23476104428082748b"} Jul 22 16:14:58.169824 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:14:58.169792 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" event={"ID":"8758c958-77f7-4174-b997-3747665753a3","Type":"ContainerStarted","Data":"96ca81d1cd031aca12dd5cb4ef283aeab1003b65244756c2a801fc771db70370"} Jul 22 16:15:00.181981 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:00.181898 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" event={"ID":"8758c958-77f7-4174-b997-3747665753a3","Type":"ContainerStarted","Data":"9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915"} Jul 22 16:15:00.181981 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:00.181930 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" podUID="8758c958-77f7-4174-b997-3747665753a3" containerName="registry-server" containerID="cri-o://9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915" gracePeriod=2 Jul 22 16:15:00.183943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:00.183918 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" event={"ID":"95bb3861-c7b6-44eb-8976-a2c62964e1ad","Type":"ContainerStarted","Data":"7a635fbcfbff7cff6ae259a356e6e168c53d0806639ca2483bc4b04175bff52c"} Jul 22 16:15:00.204597 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:00.204552 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" podStartSLOduration=1.121925432 podStartE2EDuration="3.204533512s" totalImagesPullingTime="2.08260808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:15:00.202764481 +0000 UTC m=+463.598306314" watchObservedRunningTime="2026-07-22 16:15:00.204533512 +0000 UTC m=+463.600075345" Jul 22 16:15:00.234729 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:00.234674 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" podStartSLOduration=1.6463947540000001 podStartE2EDuration="3.234657061s" totalImagesPullingTime="1.588262307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:15:00.23360771 +0000 UTC m=+463.629149545" watchObservedRunningTime="2026-07-22 16:15:00.234657061 +0000 UTC m=+463.630198896" Jul 22 16:15:00.428412 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:00.428388 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" Jul 22 16:15:00.544500 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:00.544462 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qk2g\" (UniqueName: \"kubernetes.io/projected/8758c958-77f7-4174-b997-3747665753a3-kube-api-access-5qk2g\") pod \"8758c958-77f7-4174-b997-3747665753a3\" (UID: \"8758c958-77f7-4174-b997-3747665753a3\") " Jul 22 16:15:00.546737 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:00.546711 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8758c958-77f7-4174-b997-3747665753a3-kube-api-access-5qk2g" (OuterVolumeSpecName: "kube-api-access-5qk2g") pod "8758c958-77f7-4174-b997-3747665753a3" (UID: "8758c958-77f7-4174-b997-3747665753a3"). InnerVolumeSpecName "kube-api-access-5qk2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:15:00.645474 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:00.645430 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qk2g\" (UniqueName: \"kubernetes.io/projected/8758c958-77f7-4174-b997-3747665753a3-kube-api-access-5qk2g\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:01.193607 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:01.193564 2564 generic.go:356] "Generic (PLEG): container finished" podID="8758c958-77f7-4174-b997-3747665753a3" containerID="9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915" exitCode=0 Jul 22 16:15:01.194074 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:01.193615 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" Jul 22 16:15:01.194074 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:01.193691 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" event={"ID":"8758c958-77f7-4174-b997-3747665753a3","Type":"ContainerDied","Data":"9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915"} Jul 22 16:15:01.194074 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:01.193767 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fgn4b" event={"ID":"8758c958-77f7-4174-b997-3747665753a3","Type":"ContainerDied","Data":"96ca81d1cd031aca12dd5cb4ef283aeab1003b65244756c2a801fc771db70370"} Jul 22 16:15:01.194074 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:01.193787 2564 scope.go:117] "RemoveContainer" containerID="9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915" Jul 22 16:15:01.202875 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:01.202788 2564 scope.go:117] "RemoveContainer" containerID="9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915" Jul 22 16:15:01.203096 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:01.203067 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915\": container with ID starting with 9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915 not found: ID does not exist" containerID="9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915" Jul 22 16:15:01.203180 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:01.203101 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915"} err="failed to get container status \"9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915\": rpc error: code = NotFound desc = could not find container \"9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915\": container with ID starting with 9fcc5b70b8bd7ee4dc68b30ea4e141964f38129fa2c9469301729a4395fea915 not found: ID does not exist" Jul 22 16:15:01.214841 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:01.214814 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fgn4b"] Jul 22 16:15:01.218579 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:01.218551 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fgn4b"] Jul 22 16:15:03.128756 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:03.128720 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8758c958-77f7-4174-b997-3747665753a3" path="/var/lib/kubelet/pods/8758c958-77f7-4174-b997-3747665753a3/volumes" Jul 22 16:15:07.965387 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:07.965331 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" Jul 22 16:15:07.965837 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:07.965411 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" Jul 22 16:15:07.989111 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:07.989083 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" Jul 22 16:15:08.250407 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:08.250376 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-gjrpl" Jul 22 16:15:11.621773 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.621739 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-ff8c64968-68mtq"] Jul 22 16:15:11.622141 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.622081 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8758c958-77f7-4174-b997-3747665753a3" containerName="registry-server" Jul 22 16:15:11.622141 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.622092 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8758c958-77f7-4174-b997-3747665753a3" containerName="registry-server" Jul 22 16:15:11.622228 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.622165 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8758c958-77f7-4174-b997-3747665753a3" containerName="registry-server" Jul 22 16:15:11.624246 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.624225 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.638980 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.638951 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ff8c64968-68mtq"] Jul 22 16:15:11.743422 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.743384 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fxm8\" (UniqueName: \"kubernetes.io/projected/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-kube-api-access-4fxm8\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.743618 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.743431 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-console-config\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.743618 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.743452 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-trusted-ca-bundle\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.743618 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.743548 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-console-serving-cert\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.743618 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.743588 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-service-ca\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.743840 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.743675 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-oauth-serving-cert\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.743840 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.743713 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-console-oauth-config\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.844465 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.844415 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-console-config\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.844465 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.844460 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-trusted-ca-bundle\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.844789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.844515 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-console-serving-cert\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.844789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.844545 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-service-ca\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.844789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.844586 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-oauth-serving-cert\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.844789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.844608 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-console-oauth-config\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.844789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.844698 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fxm8\" (UniqueName: \"kubernetes.io/projected/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-kube-api-access-4fxm8\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.845348 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.845314 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-console-config\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.845348 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.845330 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-service-ca\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.845526 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.845351 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-oauth-serving-cert\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.845526 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.845455 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-trusted-ca-bundle\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.847206 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.847175 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-console-serving-cert\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.847206 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.847197 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-console-oauth-config\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.854344 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.854296 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fxm8\" (UniqueName: \"kubernetes.io/projected/fa87156c-4e62-47e8-a6fb-7acbf0724dfc-kube-api-access-4fxm8\") pod \"console-ff8c64968-68mtq\" (UID: \"fa87156c-4e62-47e8-a6fb-7acbf0724dfc\") " pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:11.933099 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:11.932991 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:12.084444 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.084408 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ff8c64968-68mtq"] Jul 22 16:15:12.088276 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:15:12.088220 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa87156c_4e62_47e8_a6fb_7acbf0724dfc.slice/crio-b0cb469b432fc3d4a80d2dcd03dd0373bade0054fa430222ec075069db277401 WatchSource:0}: Error finding container b0cb469b432fc3d4a80d2dcd03dd0373bade0054fa430222ec075069db277401: Status 404 returned error can't find the container with id b0cb469b432fc3d4a80d2dcd03dd0373bade0054fa430222ec075069db277401 Jul 22 16:15:12.091257 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.091232 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f"] Jul 22 16:15:12.094894 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.094872 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.097503 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.097484 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vckdj\"" Jul 22 16:15:12.107444 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.107414 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f"] Jul 22 16:15:12.147722 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.147689 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.147902 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.147756 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmsxj\" (UniqueName: \"kubernetes.io/projected/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-kube-api-access-lmsxj\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.147902 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.147806 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.248273 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.248241 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmsxj\" (UniqueName: \"kubernetes.io/projected/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-kube-api-access-lmsxj\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.248273 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.248280 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.248562 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.248416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.248764 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.248718 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.248868 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.248846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.249152 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.249127 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff8c64968-68mtq" event={"ID":"fa87156c-4e62-47e8-a6fb-7acbf0724dfc","Type":"ContainerStarted","Data":"ed122374e23aaed8d8f7a2e4ba3e9a0ddd0caa24e424d2fcfaf0b45bc2dfcb93"} Jul 22 16:15:12.249223 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.249181 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff8c64968-68mtq" event={"ID":"fa87156c-4e62-47e8-a6fb-7acbf0724dfc","Type":"ContainerStarted","Data":"b0cb469b432fc3d4a80d2dcd03dd0373bade0054fa430222ec075069db277401"} Jul 22 16:15:12.258377 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.258350 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmsxj\" (UniqueName: \"kubernetes.io/projected/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-kube-api-access-lmsxj\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.270972 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.270921 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-ff8c64968-68mtq" podStartSLOduration=1.270905298 podStartE2EDuration="1.270905298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:15:12.269597555 +0000 UTC m=+475.665139390" watchObservedRunningTime="2026-07-22 16:15:12.270905298 +0000 UTC m=+475.666447132" Jul 22 16:15:12.407327 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.407287 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:12.544986 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.544954 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f"] Jul 22 16:15:12.547443 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:15:12.547406 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36e4e78f_4e20_49a2_ba3b_6e30a0e242e1.slice/crio-e0d7af011e4ad0c305e4501e5519bc69ab2945c1a7af2b2e4fe779620d7c8a4e WatchSource:0}: Error finding container e0d7af011e4ad0c305e4501e5519bc69ab2945c1a7af2b2e4fe779620d7c8a4e: Status 404 returned error can't find the container with id e0d7af011e4ad0c305e4501e5519bc69ab2945c1a7af2b2e4fe779620d7c8a4e Jul 22 16:15:12.683987 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.683947 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw"] Jul 22 16:15:12.686897 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.686871 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:12.701014 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.700981 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw"] Jul 22 16:15:12.753903 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.753867 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:12.754065 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.753912 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkckc\" (UniqueName: \"kubernetes.io/projected/4bfb9882-5438-47e4-845e-855a85339297-kube-api-access-fkckc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:12.754065 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.753996 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:12.854993 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.854896 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:12.854993 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.854984 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:12.855180 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.855014 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkckc\" (UniqueName: \"kubernetes.io/projected/4bfb9882-5438-47e4-845e-855a85339297-kube-api-access-fkckc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:12.855380 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.855354 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:12.855419 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.855365 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:12.865215 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.865190 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkckc\" (UniqueName: \"kubernetes.io/projected/4bfb9882-5438-47e4-845e-855a85339297-kube-api-access-fkckc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:12.995912 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:12.995874 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:13.134870 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.134844 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw"] Jul 22 16:15:13.137190 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:15:13.137163 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bfb9882_5438_47e4_845e_855a85339297.slice/crio-a4eb56ccda5236f862d7d3b7776eb33db633fd379a2df90b25eb37bfa80563e2 WatchSource:0}: Error finding container a4eb56ccda5236f862d7d3b7776eb33db633fd379a2df90b25eb37bfa80563e2: Status 404 returned error can't find the container with id a4eb56ccda5236f862d7d3b7776eb33db633fd379a2df90b25eb37bfa80563e2 Jul 22 16:15:13.256349 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.256313 2564 generic.go:356] "Generic (PLEG): container finished" podID="36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" containerID="8291ea29e9506346a1525454d868fb093a1ffbd65bcf27dfb18f975bee524ef9" exitCode=0 Jul 22 16:15:13.256520 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.256389 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" event={"ID":"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1","Type":"ContainerDied","Data":"8291ea29e9506346a1525454d868fb093a1ffbd65bcf27dfb18f975bee524ef9"} Jul 22 16:15:13.256520 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.256448 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" event={"ID":"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1","Type":"ContainerStarted","Data":"e0d7af011e4ad0c305e4501e5519bc69ab2945c1a7af2b2e4fe779620d7c8a4e"} Jul 22 16:15:13.258128 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.258101 2564 generic.go:356] "Generic (PLEG): container finished" podID="4bfb9882-5438-47e4-845e-855a85339297" containerID="e6341ac5a35b2c3821dc5169586901c3d0854f6ed506b328497e9ff41b6575ae" exitCode=0 Jul 22 16:15:13.258223 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.258183 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" event={"ID":"4bfb9882-5438-47e4-845e-855a85339297","Type":"ContainerDied","Data":"e6341ac5a35b2c3821dc5169586901c3d0854f6ed506b328497e9ff41b6575ae"} Jul 22 16:15:13.258280 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.258239 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" event={"ID":"4bfb9882-5438-47e4-845e-855a85339297","Type":"ContainerStarted","Data":"a4eb56ccda5236f862d7d3b7776eb33db633fd379a2df90b25eb37bfa80563e2"} Jul 22 16:15:13.275935 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.275906 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr"] Jul 22 16:15:13.278849 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.278780 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.290071 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.290041 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr"] Jul 22 16:15:13.358390 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.358349 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.358762 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.358729 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.358883 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.358785 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fwnx\" (UniqueName: \"kubernetes.io/projected/29b4435e-844a-4d1f-8755-b7409154f43c-kube-api-access-7fwnx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.459605 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.459493 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.459605 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.459548 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fwnx\" (UniqueName: \"kubernetes.io/projected/29b4435e-844a-4d1f-8755-b7409154f43c-kube-api-access-7fwnx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.459605 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.459584 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.459921 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.459901 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.459984 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.459963 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.470860 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.470829 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fwnx\" (UniqueName: \"kubernetes.io/projected/29b4435e-844a-4d1f-8755-b7409154f43c-kube-api-access-7fwnx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.605166 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.605124 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:13.675781 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.675751 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k"] Jul 22 16:15:13.679452 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.679426 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:13.691860 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.691823 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k"] Jul 22 16:15:13.750068 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.750038 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr"] Jul 22 16:15:13.751948 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:15:13.751924 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b4435e_844a_4d1f_8755_b7409154f43c.slice/crio-12738cafb7ab2210ff433f9d72c689bf127227184ad7a7baf368e097814992d3 WatchSource:0}: Error finding container 12738cafb7ab2210ff433f9d72c689bf127227184ad7a7baf368e097814992d3: Status 404 returned error can't find the container with id 12738cafb7ab2210ff433f9d72c689bf127227184ad7a7baf368e097814992d3 Jul 22 16:15:13.763041 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.763013 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:13.763146 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.763061 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:13.763146 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.763085 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-296lg\" (UniqueName: \"kubernetes.io/projected/6bf3a98c-b189-498c-a784-14b7a5c16199-kube-api-access-296lg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:13.864071 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.864035 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:13.864241 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.864098 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:13.864241 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.864128 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-296lg\" (UniqueName: \"kubernetes.io/projected/6bf3a98c-b189-498c-a784-14b7a5c16199-kube-api-access-296lg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:13.864533 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.864461 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:13.864533 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.864502 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:13.874324 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.874297 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-296lg\" (UniqueName: \"kubernetes.io/projected/6bf3a98c-b189-498c-a784-14b7a5c16199-kube-api-access-296lg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:13.990983 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:13.990958 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:14.138150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:14.138053 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k"] Jul 22 16:15:14.158359 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:15:14.158315 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf3a98c_b189_498c_a784_14b7a5c16199.slice/crio-257d02838a989ae841b8ce513f1bafa3f524717bf6819528ff327b37d67506b6 WatchSource:0}: Error finding container 257d02838a989ae841b8ce513f1bafa3f524717bf6819528ff327b37d67506b6: Status 404 returned error can't find the container with id 257d02838a989ae841b8ce513f1bafa3f524717bf6819528ff327b37d67506b6 Jul 22 16:15:14.267946 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:14.267912 2564 generic.go:356] "Generic (PLEG): container finished" podID="29b4435e-844a-4d1f-8755-b7409154f43c" containerID="e9c03c7049f6e26be298b3d5c76b2975882d2080ac905d92ff11db49321d9217" exitCode=0 Jul 22 16:15:14.268091 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:14.267994 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" event={"ID":"29b4435e-844a-4d1f-8755-b7409154f43c","Type":"ContainerDied","Data":"e9c03c7049f6e26be298b3d5c76b2975882d2080ac905d92ff11db49321d9217"} Jul 22 16:15:14.268091 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:14.268045 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" event={"ID":"29b4435e-844a-4d1f-8755-b7409154f43c","Type":"ContainerStarted","Data":"12738cafb7ab2210ff433f9d72c689bf127227184ad7a7baf368e097814992d3"} Jul 22 16:15:14.269447 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:14.269426 2564 generic.go:356] "Generic (PLEG): container finished" podID="6bf3a98c-b189-498c-a784-14b7a5c16199" containerID="6922c14ceb335b591ddf01e5edb75d04f992405fbd09d7ebb668962d6ee4d183" exitCode=0 Jul 22 16:15:14.269530 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:14.269502 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" event={"ID":"6bf3a98c-b189-498c-a784-14b7a5c16199","Type":"ContainerDied","Data":"6922c14ceb335b591ddf01e5edb75d04f992405fbd09d7ebb668962d6ee4d183"} Jul 22 16:15:14.269577 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:14.269544 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" event={"ID":"6bf3a98c-b189-498c-a784-14b7a5c16199","Type":"ContainerStarted","Data":"257d02838a989ae841b8ce513f1bafa3f524717bf6819528ff327b37d67506b6"} Jul 22 16:15:14.271520 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:14.271497 2564 generic.go:356] "Generic (PLEG): container finished" podID="36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" containerID="3b6e74635d81094a6455ad06436f1ee05d6e92abe0c4f6d629712a1a7c8af745" exitCode=0 Jul 22 16:15:14.271658 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:14.271544 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" event={"ID":"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1","Type":"ContainerDied","Data":"3b6e74635d81094a6455ad06436f1ee05d6e92abe0c4f6d629712a1a7c8af745"} Jul 22 16:15:15.278443 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:15.278409 2564 generic.go:356] "Generic (PLEG): container finished" podID="36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" containerID="f42bafda6b2f071fd103e38f74c40fecfc1788c15f473fcc8478626a340587b7" exitCode=0 Jul 22 16:15:15.278884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:15.278491 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" event={"ID":"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1","Type":"ContainerDied","Data":"f42bafda6b2f071fd103e38f74c40fecfc1788c15f473fcc8478626a340587b7"} Jul 22 16:15:15.280434 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:15.280399 2564 generic.go:356] "Generic (PLEG): container finished" podID="4bfb9882-5438-47e4-845e-855a85339297" containerID="91d1f094578930e6a196afa498520f5314d5e063cadcf1fe4e4a5482f40e2ac5" exitCode=0 Jul 22 16:15:15.280560 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:15.280449 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" event={"ID":"4bfb9882-5438-47e4-845e-855a85339297","Type":"ContainerDied","Data":"91d1f094578930e6a196afa498520f5314d5e063cadcf1fe4e4a5482f40e2ac5"} Jul 22 16:15:16.289885 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.289851 2564 generic.go:356] "Generic (PLEG): container finished" podID="6bf3a98c-b189-498c-a784-14b7a5c16199" containerID="7e1f3861312d84725971f98c8e042a51d775514938e2b429c9c1d0ff399b51d7" exitCode=0 Jul 22 16:15:16.290303 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.289940 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" event={"ID":"6bf3a98c-b189-498c-a784-14b7a5c16199","Type":"ContainerDied","Data":"7e1f3861312d84725971f98c8e042a51d775514938e2b429c9c1d0ff399b51d7"} Jul 22 16:15:16.291907 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.291887 2564 generic.go:356] "Generic (PLEG): container finished" podID="29b4435e-844a-4d1f-8755-b7409154f43c" containerID="87ba1c4aa6ba8eb3b925b614879c4ac0f1c9ecfcbc72b93cc3950a91b5cdee20" exitCode=0 Jul 22 16:15:16.291996 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.291974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" event={"ID":"29b4435e-844a-4d1f-8755-b7409154f43c","Type":"ContainerDied","Data":"87ba1c4aa6ba8eb3b925b614879c4ac0f1c9ecfcbc72b93cc3950a91b5cdee20"} Jul 22 16:15:16.294146 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.294111 2564 generic.go:356] "Generic (PLEG): container finished" podID="4bfb9882-5438-47e4-845e-855a85339297" containerID="343acbfceea2d4bafde1d2576384a78a59d5b8a0ddd18db51cea51fa3e255480" exitCode=0 Jul 22 16:15:16.294217 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.294176 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" event={"ID":"4bfb9882-5438-47e4-845e-855a85339297","Type":"ContainerDied","Data":"343acbfceea2d4bafde1d2576384a78a59d5b8a0ddd18db51cea51fa3e255480"} Jul 22 16:15:16.443269 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.443242 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:16.588381 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.588279 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-bundle\") pod \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " Jul 22 16:15:16.588381 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.588339 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmsxj\" (UniqueName: \"kubernetes.io/projected/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-kube-api-access-lmsxj\") pod \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " Jul 22 16:15:16.588381 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.588371 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-util\") pod \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\" (UID: \"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1\") " Jul 22 16:15:16.588870 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.588832 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-bundle" (OuterVolumeSpecName: "bundle") pod "36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" (UID: "36e4e78f-4e20-49a2-ba3b-6e30a0e242e1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:16.590540 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.590519 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-kube-api-access-lmsxj" (OuterVolumeSpecName: "kube-api-access-lmsxj") pod "36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" (UID: "36e4e78f-4e20-49a2-ba3b-6e30a0e242e1"). InnerVolumeSpecName "kube-api-access-lmsxj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:15:16.593776 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.593743 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-util" (OuterVolumeSpecName: "util") pod "36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" (UID: "36e4e78f-4e20-49a2-ba3b-6e30a0e242e1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:16.689910 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.689861 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-bundle\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:16.689910 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.689904 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmsxj\" (UniqueName: \"kubernetes.io/projected/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-kube-api-access-lmsxj\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:16.689910 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:16.689914 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36e4e78f-4e20-49a2-ba3b-6e30a0e242e1-util\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:17.300868 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.300837 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" Jul 22 16:15:17.300868 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.300855 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f" event={"ID":"36e4e78f-4e20-49a2-ba3b-6e30a0e242e1","Type":"ContainerDied","Data":"e0d7af011e4ad0c305e4501e5519bc69ab2945c1a7af2b2e4fe779620d7c8a4e"} Jul 22 16:15:17.301398 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.300902 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0d7af011e4ad0c305e4501e5519bc69ab2945c1a7af2b2e4fe779620d7c8a4e" Jul 22 16:15:17.303347 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.303317 2564 generic.go:356] "Generic (PLEG): container finished" podID="29b4435e-844a-4d1f-8755-b7409154f43c" containerID="0bccd5dbfe91fd766d4d06478582ade8a2182a25d2e2a03eb7e426c157540136" exitCode=0 Jul 22 16:15:17.303485 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.303403 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" event={"ID":"29b4435e-844a-4d1f-8755-b7409154f43c","Type":"ContainerDied","Data":"0bccd5dbfe91fd766d4d06478582ade8a2182a25d2e2a03eb7e426c157540136"} Jul 22 16:15:17.305449 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.305427 2564 generic.go:356] "Generic (PLEG): container finished" podID="6bf3a98c-b189-498c-a784-14b7a5c16199" containerID="2432199e70889965a49dc47ca0e995bda25cc321818088240888eedace45431c" exitCode=0 Jul 22 16:15:17.305559 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.305502 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" event={"ID":"6bf3a98c-b189-498c-a784-14b7a5c16199","Type":"ContainerDied","Data":"2432199e70889965a49dc47ca0e995bda25cc321818088240888eedace45431c"} Jul 22 16:15:17.445405 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.445380 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:17.597800 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.597700 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-bundle\") pod \"4bfb9882-5438-47e4-845e-855a85339297\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " Jul 22 16:15:17.597800 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.597754 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-util\") pod \"4bfb9882-5438-47e4-845e-855a85339297\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " Jul 22 16:15:17.597800 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.597777 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkckc\" (UniqueName: \"kubernetes.io/projected/4bfb9882-5438-47e4-845e-855a85339297-kube-api-access-fkckc\") pod \"4bfb9882-5438-47e4-845e-855a85339297\" (UID: \"4bfb9882-5438-47e4-845e-855a85339297\") " Jul 22 16:15:17.598251 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.598225 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-bundle" (OuterVolumeSpecName: "bundle") pod "4bfb9882-5438-47e4-845e-855a85339297" (UID: "4bfb9882-5438-47e4-845e-855a85339297"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:17.600230 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.600202 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bfb9882-5438-47e4-845e-855a85339297-kube-api-access-fkckc" (OuterVolumeSpecName: "kube-api-access-fkckc") pod "4bfb9882-5438-47e4-845e-855a85339297" (UID: "4bfb9882-5438-47e4-845e-855a85339297"). InnerVolumeSpecName "kube-api-access-fkckc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:15:17.603636 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.603601 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-util" (OuterVolumeSpecName: "util") pod "4bfb9882-5438-47e4-845e-855a85339297" (UID: "4bfb9882-5438-47e4-845e-855a85339297"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:17.699212 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.699166 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-bundle\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:17.699212 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.699203 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4bfb9882-5438-47e4-845e-855a85339297-util\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:17.699212 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:17.699217 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkckc\" (UniqueName: \"kubernetes.io/projected/4bfb9882-5438-47e4-845e-855a85339297-kube-api-access-fkckc\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:18.312642 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.312582 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" event={"ID":"4bfb9882-5438-47e4-845e-855a85339297","Type":"ContainerDied","Data":"a4eb56ccda5236f862d7d3b7776eb33db633fd379a2df90b25eb37bfa80563e2"} Jul 22 16:15:18.313137 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.312664 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4eb56ccda5236f862d7d3b7776eb33db633fd379a2df90b25eb37bfa80563e2" Jul 22 16:15:18.313137 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.312605 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw" Jul 22 16:15:18.461163 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.461136 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:18.490913 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.490885 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:18.606758 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.606671 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-util\") pod \"6bf3a98c-b189-498c-a784-14b7a5c16199\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " Jul 22 16:15:18.606758 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.606716 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-296lg\" (UniqueName: \"kubernetes.io/projected/6bf3a98c-b189-498c-a784-14b7a5c16199-kube-api-access-296lg\") pod \"6bf3a98c-b189-498c-a784-14b7a5c16199\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " Jul 22 16:15:18.606758 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.606760 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-bundle\") pod \"29b4435e-844a-4d1f-8755-b7409154f43c\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " Jul 22 16:15:18.607013 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.606781 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-bundle\") pod \"6bf3a98c-b189-498c-a784-14b7a5c16199\" (UID: \"6bf3a98c-b189-498c-a784-14b7a5c16199\") " Jul 22 16:15:18.607013 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.606806 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fwnx\" (UniqueName: \"kubernetes.io/projected/29b4435e-844a-4d1f-8755-b7409154f43c-kube-api-access-7fwnx\") pod \"29b4435e-844a-4d1f-8755-b7409154f43c\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " Jul 22 16:15:18.607013 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.606825 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-util\") pod \"29b4435e-844a-4d1f-8755-b7409154f43c\" (UID: \"29b4435e-844a-4d1f-8755-b7409154f43c\") " Jul 22 16:15:18.607433 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.607400 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-bundle" (OuterVolumeSpecName: "bundle") pod "29b4435e-844a-4d1f-8755-b7409154f43c" (UID: "29b4435e-844a-4d1f-8755-b7409154f43c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:18.607568 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.607516 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-bundle" (OuterVolumeSpecName: "bundle") pod "6bf3a98c-b189-498c-a784-14b7a5c16199" (UID: "6bf3a98c-b189-498c-a784-14b7a5c16199"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:18.609061 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.609035 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf3a98c-b189-498c-a784-14b7a5c16199-kube-api-access-296lg" (OuterVolumeSpecName: "kube-api-access-296lg") pod "6bf3a98c-b189-498c-a784-14b7a5c16199" (UID: "6bf3a98c-b189-498c-a784-14b7a5c16199"). InnerVolumeSpecName "kube-api-access-296lg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:15:18.609243 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.609229 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29b4435e-844a-4d1f-8755-b7409154f43c-kube-api-access-7fwnx" (OuterVolumeSpecName: "kube-api-access-7fwnx") pod "29b4435e-844a-4d1f-8755-b7409154f43c" (UID: "29b4435e-844a-4d1f-8755-b7409154f43c"). InnerVolumeSpecName "kube-api-access-7fwnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:15:18.613182 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.613148 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-util" (OuterVolumeSpecName: "util") pod "6bf3a98c-b189-498c-a784-14b7a5c16199" (UID: "6bf3a98c-b189-498c-a784-14b7a5c16199"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:18.615280 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.615249 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-util" (OuterVolumeSpecName: "util") pod "29b4435e-844a-4d1f-8755-b7409154f43c" (UID: "29b4435e-844a-4d1f-8755-b7409154f43c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:18.707644 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.707582 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-bundle\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:18.707644 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.707616 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-bundle\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:18.707644 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.707649 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fwnx\" (UniqueName: \"kubernetes.io/projected/29b4435e-844a-4d1f-8755-b7409154f43c-kube-api-access-7fwnx\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:18.707957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.707660 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29b4435e-844a-4d1f-8755-b7409154f43c-util\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:18.707957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.707670 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bf3a98c-b189-498c-a784-14b7a5c16199-util\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:18.707957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:18.707678 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-296lg\" (UniqueName: \"kubernetes.io/projected/6bf3a98c-b189-498c-a784-14b7a5c16199-kube-api-access-296lg\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:19.140863 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:19.140828 2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b4435e_844a_4d1f_8755_b7409154f43c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b4435e_844a_4d1f_8755_b7409154f43c.slice/crio-12738cafb7ab2210ff433f9d72c689bf127227184ad7a7baf368e097814992d3\": RecentStats: unable to find data in memory cache]" Jul 22 16:15:19.319118 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:19.319082 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" Jul 22 16:15:19.319118 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:19.319104 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k" event={"ID":"6bf3a98c-b189-498c-a784-14b7a5c16199","Type":"ContainerDied","Data":"257d02838a989ae841b8ce513f1bafa3f524717bf6819528ff327b37d67506b6"} Jul 22 16:15:19.319615 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:19.319153 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="257d02838a989ae841b8ce513f1bafa3f524717bf6819528ff327b37d67506b6" Jul 22 16:15:19.321290 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:19.321260 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" event={"ID":"29b4435e-844a-4d1f-8755-b7409154f43c","Type":"ContainerDied","Data":"12738cafb7ab2210ff433f9d72c689bf127227184ad7a7baf368e097814992d3"} Jul 22 16:15:19.321420 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:19.321307 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12738cafb7ab2210ff433f9d72c689bf127227184ad7a7baf368e097814992d3" Jul 22 16:15:19.321420 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:19.321315 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr" Jul 22 16:15:21.933812 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:21.933770 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:21.934319 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:21.933922 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:21.938805 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:21.938777 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:22.340321 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:22.340287 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-ff8c64968-68mtq" Jul 22 16:15:22.398237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:22.398203 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f55d47cbf-kz2nq"] Jul 22 16:15:29.643069 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643032 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cckvl"] Jul 22 16:15:29.643584 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643561 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29b4435e-844a-4d1f-8755-b7409154f43c" containerName="pull" Jul 22 16:15:29.643584 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643581 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29b4435e-844a-4d1f-8755-b7409154f43c" containerName="pull" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643595 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" containerName="pull" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643603 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" containerName="pull" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643618 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29b4435e-844a-4d1f-8755-b7409154f43c" containerName="util" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643654 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29b4435e-844a-4d1f-8755-b7409154f43c" containerName="util" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643672 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bfb9882-5438-47e4-845e-855a85339297" containerName="extract" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643681 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bfb9882-5438-47e4-845e-855a85339297" containerName="extract" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643692 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29b4435e-844a-4d1f-8755-b7409154f43c" containerName="extract" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643700 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="29b4435e-844a-4d1f-8755-b7409154f43c" containerName="extract" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643714 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf3a98c-b189-498c-a784-14b7a5c16199" containerName="extract" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643722 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf3a98c-b189-498c-a784-14b7a5c16199" containerName="extract" Jul 22 16:15:29.643739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643738 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bfb9882-5438-47e4-845e-855a85339297" containerName="pull" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643746 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bfb9882-5438-47e4-845e-855a85339297" containerName="pull" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643757 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" containerName="extract" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643766 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" containerName="extract" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643776 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf3a98c-b189-498c-a784-14b7a5c16199" containerName="util" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643784 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf3a98c-b189-498c-a784-14b7a5c16199" containerName="util" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643795 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf3a98c-b189-498c-a784-14b7a5c16199" containerName="pull" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643803 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf3a98c-b189-498c-a784-14b7a5c16199" containerName="pull" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643818 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" containerName="util" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643827 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" containerName="util" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643844 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bfb9882-5438-47e4-845e-855a85339297" containerName="util" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643852 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bfb9882-5438-47e4-845e-855a85339297" containerName="util" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643953 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf3a98c-b189-498c-a784-14b7a5c16199" containerName="extract" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643970 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="36e4e78f-4e20-49a2-ba3b-6e30a0e242e1" containerName="extract" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643981 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bfb9882-5438-47e4-845e-855a85339297" containerName="extract" Jul 22 16:15:29.644237 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.643989 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="29b4435e-844a-4d1f-8755-b7409154f43c" containerName="extract" Jul 22 16:15:29.649012 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.648988 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cckvl" Jul 22 16:15:29.652797 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.652772 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ppb4c\"" Jul 22 16:15:29.664383 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.664357 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cckvl"] Jul 22 16:15:29.812939 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.812894 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t9zv\" (UniqueName: \"kubernetes.io/projected/704d3a3e-ecfc-4dfe-8f2e-277902356b20-kube-api-access-6t9zv\") pod \"authorino-operator-75cc8d5cb7-cckvl\" (UID: \"704d3a3e-ecfc-4dfe-8f2e-277902356b20\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cckvl" Jul 22 16:15:29.914271 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.914156 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t9zv\" (UniqueName: \"kubernetes.io/projected/704d3a3e-ecfc-4dfe-8f2e-277902356b20-kube-api-access-6t9zv\") pod \"authorino-operator-75cc8d5cb7-cckvl\" (UID: \"704d3a3e-ecfc-4dfe-8f2e-277902356b20\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cckvl" Jul 22 16:15:29.927401 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.927364 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t9zv\" (UniqueName: \"kubernetes.io/projected/704d3a3e-ecfc-4dfe-8f2e-277902356b20-kube-api-access-6t9zv\") pod \"authorino-operator-75cc8d5cb7-cckvl\" (UID: \"704d3a3e-ecfc-4dfe-8f2e-277902356b20\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cckvl" Jul 22 16:15:29.959475 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:29.959435 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cckvl" Jul 22 16:15:30.108729 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:30.108700 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cckvl"] Jul 22 16:15:30.110611 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:15:30.110583 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod704d3a3e_ecfc_4dfe_8f2e_277902356b20.slice/crio-68479218cc91c3f2ab19a82b878d6538b961b678ded160e3e0860ae358847527 WatchSource:0}: Error finding container 68479218cc91c3f2ab19a82b878d6538b961b678ded160e3e0860ae358847527: Status 404 returned error can't find the container with id 68479218cc91c3f2ab19a82b878d6538b961b678ded160e3e0860ae358847527 Jul 22 16:15:30.375421 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:30.375380 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cckvl" event={"ID":"704d3a3e-ecfc-4dfe-8f2e-277902356b20","Type":"ContainerStarted","Data":"68479218cc91c3f2ab19a82b878d6538b961b678ded160e3e0860ae358847527"} Jul 22 16:15:32.469601 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:32.469558 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6"] Jul 22 16:15:32.479997 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:32.479965 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:15:32.483870 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:32.483700 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vrg4s\"" Jul 22 16:15:32.490058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:32.490017 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6"] Jul 22 16:15:32.639883 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:32.639842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkg2m\" (UniqueName: \"kubernetes.io/projected/639a1187-e723-40f9-8f8a-beb5d27889d6-kube-api-access-dkg2m\") pod \"limitador-operator-controller-manager-859d7b78bd-rsvf6\" (UID: \"639a1187-e723-40f9-8f8a-beb5d27889d6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:15:32.741091 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:32.741045 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkg2m\" (UniqueName: \"kubernetes.io/projected/639a1187-e723-40f9-8f8a-beb5d27889d6-kube-api-access-dkg2m\") pod \"limitador-operator-controller-manager-859d7b78bd-rsvf6\" (UID: \"639a1187-e723-40f9-8f8a-beb5d27889d6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:15:32.757619 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:32.757581 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkg2m\" (UniqueName: \"kubernetes.io/projected/639a1187-e723-40f9-8f8a-beb5d27889d6-kube-api-access-dkg2m\") pod \"limitador-operator-controller-manager-859d7b78bd-rsvf6\" (UID: \"639a1187-e723-40f9-8f8a-beb5d27889d6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:15:32.794330 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:32.794283 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:15:33.166192 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:33.166170 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6"] Jul 22 16:15:33.168258 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:15:33.168228 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639a1187_e723_40f9_8f8a_beb5d27889d6.slice/crio-a2ae08ee109268b724f3b310dda82aba8d50f505accab9b2eca97af392ab64ee WatchSource:0}: Error finding container a2ae08ee109268b724f3b310dda82aba8d50f505accab9b2eca97af392ab64ee: Status 404 returned error can't find the container with id a2ae08ee109268b724f3b310dda82aba8d50f505accab9b2eca97af392ab64ee Jul 22 16:15:33.399725 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:33.399596 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cckvl" event={"ID":"704d3a3e-ecfc-4dfe-8f2e-277902356b20","Type":"ContainerStarted","Data":"f720b1123c85d06d7216a7585e91a57889a625e889e955845a61568d04300c51"} Jul 22 16:15:33.399897 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:33.399766 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cckvl" Jul 22 16:15:33.400952 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:33.400927 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" event={"ID":"639a1187-e723-40f9-8f8a-beb5d27889d6","Type":"ContainerStarted","Data":"a2ae08ee109268b724f3b310dda82aba8d50f505accab9b2eca97af392ab64ee"} Jul 22 16:15:33.427227 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:33.427155 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cckvl" podStartSLOduration=1.469778142 podStartE2EDuration="4.427138695s" totalImagesPullingTime="2.957360553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:15:33.425417159 +0000 UTC m=+496.820958997" watchObservedRunningTime="2026-07-22 16:15:33.427138695 +0000 UTC m=+496.822680606" Jul 22 16:15:35.144308 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.144229 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff"] Jul 22 16:15:35.147183 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.147164 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff" Jul 22 16:15:35.150577 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.150555 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 22 16:15:35.150982 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.150966 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5bltx\"" Jul 22 16:15:35.168891 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.168859 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff"] Jul 22 16:15:35.264749 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.264712 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvpml\" (UniqueName: \"kubernetes.io/projected/5ea66736-bff9-4bec-9fe5-7b4842433781-kube-api-access-qvpml\") pod \"dns-operator-controller-manager-864dd78c4c-jcqff\" (UID: \"5ea66736-bff9-4bec-9fe5-7b4842433781\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff" Jul 22 16:15:35.366043 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.365996 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvpml\" (UniqueName: \"kubernetes.io/projected/5ea66736-bff9-4bec-9fe5-7b4842433781-kube-api-access-qvpml\") pod \"dns-operator-controller-manager-864dd78c4c-jcqff\" (UID: \"5ea66736-bff9-4bec-9fe5-7b4842433781\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff" Jul 22 16:15:35.387079 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.387041 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvpml\" (UniqueName: \"kubernetes.io/projected/5ea66736-bff9-4bec-9fe5-7b4842433781-kube-api-access-qvpml\") pod \"dns-operator-controller-manager-864dd78c4c-jcqff\" (UID: \"5ea66736-bff9-4bec-9fe5-7b4842433781\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff" Jul 22 16:15:35.419500 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.419390 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" event={"ID":"639a1187-e723-40f9-8f8a-beb5d27889d6","Type":"ContainerStarted","Data":"a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408"} Jul 22 16:15:35.419500 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.419493 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:15:35.457608 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.457570 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff" Jul 22 16:15:35.465657 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.465593 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" podStartSLOduration=1.894141351 podStartE2EDuration="3.465576251s" totalImagesPullingTime="1.5714349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:15:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:15:35.463839893 +0000 UTC m=+498.859381727" watchObservedRunningTime="2026-07-22 16:15:35.465576251 +0000 UTC m=+498.861118085" Jul 22 16:15:35.627242 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:35.627204 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff"] Jul 22 16:15:35.630726 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:15:35.630697 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ea66736_bff9_4bec_9fe5_7b4842433781.slice/crio-88e222e420f394afecc7d7508be86bfbb2ecc9ecd0634fe73756cd83bcd80be4 WatchSource:0}: Error finding container 88e222e420f394afecc7d7508be86bfbb2ecc9ecd0634fe73756cd83bcd80be4: Status 404 returned error can't find the container with id 88e222e420f394afecc7d7508be86bfbb2ecc9ecd0634fe73756cd83bcd80be4 Jul 22 16:15:36.425551 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:36.425507 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff" event={"ID":"5ea66736-bff9-4bec-9fe5-7b4842433781","Type":"ContainerStarted","Data":"88e222e420f394afecc7d7508be86bfbb2ecc9ecd0634fe73756cd83bcd80be4"} Jul 22 16:15:39.326013 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.325969 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td"] Jul 22 16:15:39.329437 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.329411 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:39.333240 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.333213 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fpl8z\"" Jul 22 16:15:39.349565 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.349538 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td"] Jul 22 16:15:39.443222 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.443177 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff" event={"ID":"5ea66736-bff9-4bec-9fe5-7b4842433781","Type":"ContainerStarted","Data":"ab52bd2b8158ef98a1a462000bb91d198d005d16c42356137c8545482ca372cf"} Jul 22 16:15:39.443394 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.443363 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff" Jul 22 16:15:39.470462 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.470408 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff" podStartSLOduration=1.393744692 podStartE2EDuration="4.47039255s" totalImagesPullingTime="3.076647858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:15:39.46959813 +0000 UTC m=+502.865139965" watchObservedRunningTime="2026-07-22 16:15:39.47039255 +0000 UTC m=+502.865934383" Jul 22 16:15:39.502071 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.502031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlglz\" (UniqueName: \"kubernetes.io/projected/d133109e-430f-4439-9ce5-5a277ae059c6-kube-api-access-dlglz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" (UID: \"d133109e-430f-4439-9ce5-5a277ae059c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:39.502071 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.502071 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d133109e-430f-4439-9ce5-5a277ae059c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" (UID: \"d133109e-430f-4439-9ce5-5a277ae059c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:39.603476 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.603390 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlglz\" (UniqueName: \"kubernetes.io/projected/d133109e-430f-4439-9ce5-5a277ae059c6-kube-api-access-dlglz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" (UID: \"d133109e-430f-4439-9ce5-5a277ae059c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:39.603476 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.603447 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d133109e-430f-4439-9ce5-5a277ae059c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" (UID: \"d133109e-430f-4439-9ce5-5a277ae059c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:39.603900 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.603879 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d133109e-430f-4439-9ce5-5a277ae059c6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" (UID: \"d133109e-430f-4439-9ce5-5a277ae059c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:39.616003 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.615972 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlglz\" (UniqueName: \"kubernetes.io/projected/d133109e-430f-4439-9ce5-5a277ae059c6-kube-api-access-dlglz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" (UID: \"d133109e-430f-4439-9ce5-5a277ae059c6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:39.639725 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.639691 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:39.812228 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:39.812158 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td"] Jul 22 16:15:39.815088 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:15:39.815057 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd133109e_430f_4439_9ce5_5a277ae059c6.slice/crio-08b9e2c8379ff64d2365710bdf093fecb37ea0112c7e8de09abae245d8c3f5cf WatchSource:0}: Error finding container 08b9e2c8379ff64d2365710bdf093fecb37ea0112c7e8de09abae245d8c3f5cf: Status 404 returned error can't find the container with id 08b9e2c8379ff64d2365710bdf093fecb37ea0112c7e8de09abae245d8c3f5cf Jul 22 16:15:40.449717 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:40.449677 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" event={"ID":"d133109e-430f-4439-9ce5-5a277ae059c6","Type":"ContainerStarted","Data":"08b9e2c8379ff64d2365710bdf093fecb37ea0112c7e8de09abae245d8c3f5cf"} Jul 22 16:15:44.410288 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:44.410250 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cckvl" Jul 22 16:15:45.479133 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:45.479099 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" event={"ID":"d133109e-430f-4439-9ce5-5a277ae059c6","Type":"ContainerStarted","Data":"b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7"} Jul 22 16:15:45.479589 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:45.479241 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:45.507324 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:45.507273 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" podStartSLOduration=1.171107469 podStartE2EDuration="6.507255416s" totalImagesPullingTime="5.336147947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:15:45.505874119 +0000 UTC m=+508.901415957" watchObservedRunningTime="2026-07-22 16:15:45.507255416 +0000 UTC m=+508.902797250" Jul 22 16:15:46.427907 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:46.427866 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:15:47.426561 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.426502 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f55d47cbf-kz2nq" podUID="86d1e78c-ade2-43ed-a901-77c4daac75d7" containerName="console" containerID="cri-o://2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49" gracePeriod=15 Jul 22 16:15:47.692569 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.692505 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:15:47.780342 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780298 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-service-ca\") pod \"86d1e78c-ade2-43ed-a901-77c4daac75d7\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " Jul 22 16:15:47.780529 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780378 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-config\") pod \"86d1e78c-ade2-43ed-a901-77c4daac75d7\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " Jul 22 16:15:47.780529 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780442 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-trusted-ca-bundle\") pod \"86d1e78c-ade2-43ed-a901-77c4daac75d7\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " Jul 22 16:15:47.780529 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780469 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-oauth-serving-cert\") pod \"86d1e78c-ade2-43ed-a901-77c4daac75d7\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " Jul 22 16:15:47.780529 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780491 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5kx8\" (UniqueName: \"kubernetes.io/projected/86d1e78c-ade2-43ed-a901-77c4daac75d7-kube-api-access-k5kx8\") pod \"86d1e78c-ade2-43ed-a901-77c4daac75d7\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " Jul 22 16:15:47.780765 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780542 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-oauth-config\") pod \"86d1e78c-ade2-43ed-a901-77c4daac75d7\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " Jul 22 16:15:47.780765 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780575 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-serving-cert\") pod \"86d1e78c-ade2-43ed-a901-77c4daac75d7\" (UID: \"86d1e78c-ade2-43ed-a901-77c4daac75d7\") " Jul 22 16:15:47.780878 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780849 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-config" (OuterVolumeSpecName: "console-config") pod "86d1e78c-ade2-43ed-a901-77c4daac75d7" (UID: "86d1e78c-ade2-43ed-a901-77c4daac75d7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:47.780936 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780781 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-service-ca" (OuterVolumeSpecName: "service-ca") pod "86d1e78c-ade2-43ed-a901-77c4daac75d7" (UID: "86d1e78c-ade2-43ed-a901-77c4daac75d7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:47.780993 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780932 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "86d1e78c-ade2-43ed-a901-77c4daac75d7" (UID: "86d1e78c-ade2-43ed-a901-77c4daac75d7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:47.780993 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.780946 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "86d1e78c-ade2-43ed-a901-77c4daac75d7" (UID: "86d1e78c-ade2-43ed-a901-77c4daac75d7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:47.782931 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.782893 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "86d1e78c-ade2-43ed-a901-77c4daac75d7" (UID: "86d1e78c-ade2-43ed-a901-77c4daac75d7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:47.782931 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.782904 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86d1e78c-ade2-43ed-a901-77c4daac75d7-kube-api-access-k5kx8" (OuterVolumeSpecName: "kube-api-access-k5kx8") pod "86d1e78c-ade2-43ed-a901-77c4daac75d7" (UID: "86d1e78c-ade2-43ed-a901-77c4daac75d7"). InnerVolumeSpecName "kube-api-access-k5kx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:15:47.783092 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.782962 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "86d1e78c-ade2-43ed-a901-77c4daac75d7" (UID: "86d1e78c-ade2-43ed-a901-77c4daac75d7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:47.881861 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.881826 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-service-ca\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:47.881861 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.881855 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-config\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:47.881861 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.881864 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-trusted-ca-bundle\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:47.882119 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.881874 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86d1e78c-ade2-43ed-a901-77c4daac75d7-oauth-serving-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:47.882119 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.881883 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5kx8\" (UniqueName: \"kubernetes.io/projected/86d1e78c-ade2-43ed-a901-77c4daac75d7-kube-api-access-k5kx8\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:47.882119 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.881892 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-oauth-config\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:47.882119 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:47.881901 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86d1e78c-ade2-43ed-a901-77c4daac75d7-console-serving-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:48.057046 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.057012 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z"] Jul 22 16:15:48.057420 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.057408 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d1e78c-ade2-43ed-a901-77c4daac75d7" containerName="console" Jul 22 16:15:48.057465 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.057436 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d1e78c-ade2-43ed-a901-77c4daac75d7" containerName="console" Jul 22 16:15:48.057524 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.057515 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d1e78c-ade2-43ed-a901-77c4daac75d7" containerName="console" Jul 22 16:15:48.060943 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.060925 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.063538 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.063511 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 22 16:15:48.063785 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.063768 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vckdj\"" Jul 22 16:15:48.063968 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.063956 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 22 16:15:48.069858 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.069832 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z"] Jul 22 16:15:48.083575 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.083546 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d0d4aeb7-5c31-4e66-b082-a6f2160e7456-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nsd9z\" (UID: \"d0d4aeb7-5c31-4e66-b082-a6f2160e7456\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.083703 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.083585 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7chh\" (UniqueName: \"kubernetes.io/projected/d0d4aeb7-5c31-4e66-b082-a6f2160e7456-kube-api-access-b7chh\") pod \"kuadrant-console-plugin-66b5774985-nsd9z\" (UID: \"d0d4aeb7-5c31-4e66-b082-a6f2160e7456\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.083703 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.083666 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d4aeb7-5c31-4e66-b082-a6f2160e7456-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nsd9z\" (UID: \"d0d4aeb7-5c31-4e66-b082-a6f2160e7456\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.184125 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.184085 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d4aeb7-5c31-4e66-b082-a6f2160e7456-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nsd9z\" (UID: \"d0d4aeb7-5c31-4e66-b082-a6f2160e7456\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.184334 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.184165 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d0d4aeb7-5c31-4e66-b082-a6f2160e7456-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nsd9z\" (UID: \"d0d4aeb7-5c31-4e66-b082-a6f2160e7456\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.184334 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.184290 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7chh\" (UniqueName: \"kubernetes.io/projected/d0d4aeb7-5c31-4e66-b082-a6f2160e7456-kube-api-access-b7chh\") pod \"kuadrant-console-plugin-66b5774985-nsd9z\" (UID: \"d0d4aeb7-5c31-4e66-b082-a6f2160e7456\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.184866 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.184847 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d0d4aeb7-5c31-4e66-b082-a6f2160e7456-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nsd9z\" (UID: \"d0d4aeb7-5c31-4e66-b082-a6f2160e7456\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.186642 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.186606 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d4aeb7-5c31-4e66-b082-a6f2160e7456-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nsd9z\" (UID: \"d0d4aeb7-5c31-4e66-b082-a6f2160e7456\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.194120 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.194095 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7chh\" (UniqueName: \"kubernetes.io/projected/d0d4aeb7-5c31-4e66-b082-a6f2160e7456-kube-api-access-b7chh\") pod \"kuadrant-console-plugin-66b5774985-nsd9z\" (UID: \"d0d4aeb7-5c31-4e66-b082-a6f2160e7456\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.371249 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.371164 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" Jul 22 16:15:48.497693 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.497654 2564 generic.go:356] "Generic (PLEG): container finished" podID="86d1e78c-ade2-43ed-a901-77c4daac75d7" containerID="2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49" exitCode=0 Jul 22 16:15:48.498094 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.497709 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f55d47cbf-kz2nq" Jul 22 16:15:48.498094 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.497741 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f55d47cbf-kz2nq" event={"ID":"86d1e78c-ade2-43ed-a901-77c4daac75d7","Type":"ContainerDied","Data":"2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49"} Jul 22 16:15:48.498094 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.497794 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f55d47cbf-kz2nq" event={"ID":"86d1e78c-ade2-43ed-a901-77c4daac75d7","Type":"ContainerDied","Data":"787e211684c444385c36f35e574269908d9ba1a0676bd59313d04b31d3191d18"} Jul 22 16:15:48.498094 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.497816 2564 scope.go:117] "RemoveContainer" containerID="2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49" Jul 22 16:15:48.507802 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.507777 2564 scope.go:117] "RemoveContainer" containerID="2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49" Jul 22 16:15:48.508057 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:48.508036 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49\": container with ID starting with 2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49 not found: ID does not exist" containerID="2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49" Jul 22 16:15:48.508105 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.508068 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49"} err="failed to get container status \"2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49\": rpc error: code = NotFound desc = could not find container \"2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49\": container with ID starting with 2c8e282aab4b9220bf3b7e6136b02090ba85b305d670fefdd3b5de7233534d49 not found: ID does not exist" Jul 22 16:15:48.534842 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.534812 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f55d47cbf-kz2nq"] Jul 22 16:15:48.547760 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.547738 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z"] Jul 22 16:15:48.550301 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:15:48.550276 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0d4aeb7_5c31_4e66_b082_a6f2160e7456.slice/crio-4e32652ada0ba0606f381750673a700a1a09e6952405da84860b5f5f0b95ccab WatchSource:0}: Error finding container 4e32652ada0ba0606f381750673a700a1a09e6952405da84860b5f5f0b95ccab: Status 404 returned error can't find the container with id 4e32652ada0ba0606f381750673a700a1a09e6952405da84860b5f5f0b95ccab Jul 22 16:15:48.551098 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:48.551077 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f55d47cbf-kz2nq"] Jul 22 16:15:49.129318 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:49.129280 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86d1e78c-ade2-43ed-a901-77c4daac75d7" path="/var/lib/kubelet/pods/86d1e78c-ade2-43ed-a901-77c4daac75d7/volumes" Jul 22 16:15:49.505030 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:49.504987 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" event={"ID":"d0d4aeb7-5c31-4e66-b082-a6f2160e7456","Type":"ContainerStarted","Data":"4e32652ada0ba0606f381750673a700a1a09e6952405da84860b5f5f0b95ccab"} Jul 22 16:15:50.452715 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:50.452682 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-jcqff" Jul 22 16:15:56.488789 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:56.488752 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:58.346346 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.346306 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr"] Jul 22 16:15:58.366178 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.366147 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:15:58.371193 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.371160 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr"] Jul 22 16:15:58.484325 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.484289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48c50310-46c0-4b48-b01c-e9a057993e88-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9bntr\" (UID: \"48c50310-46c0-4b48-b01c-e9a057993e88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:15:58.484549 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.484348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsgpw\" (UniqueName: \"kubernetes.io/projected/48c50310-46c0-4b48-b01c-e9a057993e88-kube-api-access-qsgpw\") pod \"kuadrant-operator-controller-manager-86878c769f-9bntr\" (UID: \"48c50310-46c0-4b48-b01c-e9a057993e88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:15:58.527614 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.527574 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td"] Jul 22 16:15:58.527901 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.527853 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" containerName="manager" containerID="cri-o://b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7" gracePeriod=2 Jul 22 16:15:58.541331 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.541016 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td"] Jul 22 16:15:58.546211 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:58.546168 2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d133109e-430f-4439-9ce5-5a277ae059c6\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-j26td\": pods \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:58.555816 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.555785 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr"] Jul 22 16:15:58.556775 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:58.556124 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-qsgpw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" podUID="48c50310-46c0-4b48-b01c-e9a057993e88" Jul 22 16:15:58.558744 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.558718 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w"] Jul 22 16:15:58.559349 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.559328 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" containerName="manager" Jul 22 16:15:58.559435 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.559354 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" containerName="manager" Jul 22 16:15:58.559483 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.559469 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" containerName="manager" Jul 22 16:15:58.564523 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.564504 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:15:58.567018 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:58.566979 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:58.572949 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.572347 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:15:58.581518 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.581495 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr"] Jul 22 16:15:58.584461 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.584438 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w"] Jul 22 16:15:58.585072 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.585048 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48c50310-46c0-4b48-b01c-e9a057993e88-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9bntr\" (UID: \"48c50310-46c0-4b48-b01c-e9a057993e88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:15:58.585239 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.585221 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgpw\" (UniqueName: \"kubernetes.io/projected/48c50310-46c0-4b48-b01c-e9a057993e88-kube-api-access-qsgpw\") pod \"kuadrant-operator-controller-manager-86878c769f-9bntr\" (UID: \"48c50310-46c0-4b48-b01c-e9a057993e88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:15:58.585651 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.585472 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48c50310-46c0-4b48-b01c-e9a057993e88-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9bntr\" (UID: \"48c50310-46c0-4b48-b01c-e9a057993e88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:15:58.594869 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:58.594847 2564 projected.go:196] Error preparing data for projected volume kube-api-access-qsgpw for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-9bntr" not found Jul 22 16:15:58.595022 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:58.594929 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/48c50310-46c0-4b48-b01c-e9a057993e88-kube-api-access-qsgpw podName:48c50310-46c0-4b48-b01c-e9a057993e88 nodeName:}" failed. No retries permitted until 2026-07-22 16:15:59.094907235 +0000 UTC m=+522.490449051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qsgpw" (UniqueName: "kubernetes.io/projected/48c50310-46c0-4b48-b01c-e9a057993e88-kube-api-access-qsgpw") pod "kuadrant-operator-controller-manager-86878c769f-9bntr" (UID: "48c50310-46c0-4b48-b01c-e9a057993e88") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-9bntr" not found Jul 22 16:15:58.598293 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.598238 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h"] Jul 22 16:15:58.604539 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.604515 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6"] Jul 22 16:15:58.604686 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.604666 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:15:58.604922 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.604894 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" podUID="639a1187-e723-40f9-8f8a-beb5d27889d6" containerName="manager" containerID="cri-o://a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408" gracePeriod=2 Jul 22 16:15:58.605227 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.605209 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:15:58.623971 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:58.623929 2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"639a1187-e723-40f9-8f8a-beb5d27889d6\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-rsvf6\": pods \"limitador-operator-controller-manager-859d7b78bd-rsvf6\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:15:58.625084 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.625019 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6"] Jul 22 16:15:58.628043 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.628017 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h"] Jul 22 16:15:58.668155 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:58.668114 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-9bntr\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="48c50310-46c0-4b48-b01c-e9a057993e88" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:15:58.670894 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:58.670852 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:15:58.687030 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.686988 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48c50310-46c0-4b48-b01c-e9a057993e88-extensions-socket-volume\") pod \"48c50310-46c0-4b48-b01c-e9a057993e88\" (UID: \"48c50310-46c0-4b48-b01c-e9a057993e88\") " Jul 22 16:15:58.687377 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.687275 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48c50310-46c0-4b48-b01c-e9a057993e88-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "48c50310-46c0-4b48-b01c-e9a057993e88" (UID: "48c50310-46c0-4b48-b01c-e9a057993e88"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:58.687542 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.687314 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2czl\" (UniqueName: \"kubernetes.io/projected/eb6723e3-baac-40a3-bfef-aa44a9f62d64-kube-api-access-h2czl\") pod \"kuadrant-operator-controller-manager-86878c769f-bz69h\" (UID: \"eb6723e3-baac-40a3-bfef-aa44a9f62d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:15:58.687617 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.687535 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxk6h\" (UniqueName: \"kubernetes.io/projected/81826a66-3411-449f-87fe-071f65d2165b-kube-api-access-kxk6h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p784w\" (UID: \"81826a66-3411-449f-87fe-071f65d2165b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:15:58.687617 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.687598 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb6723e3-baac-40a3-bfef-aa44a9f62d64-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bz69h\" (UID: \"eb6723e3-baac-40a3-bfef-aa44a9f62d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:15:58.687766 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.687719 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81826a66-3411-449f-87fe-071f65d2165b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p784w\" (UID: \"81826a66-3411-449f-87fe-071f65d2165b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:15:58.687827 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.687778 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/48c50310-46c0-4b48-b01c-e9a057993e88-extensions-socket-volume\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:58.687827 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.687789 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsgpw\" (UniqueName: \"kubernetes.io/projected/48c50310-46c0-4b48-b01c-e9a057993e88-kube-api-access-qsgpw\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:15:58.788707 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.788668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2czl\" (UniqueName: \"kubernetes.io/projected/eb6723e3-baac-40a3-bfef-aa44a9f62d64-kube-api-access-h2czl\") pod \"kuadrant-operator-controller-manager-86878c769f-bz69h\" (UID: \"eb6723e3-baac-40a3-bfef-aa44a9f62d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:15:58.788899 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.788729 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxk6h\" (UniqueName: \"kubernetes.io/projected/81826a66-3411-449f-87fe-071f65d2165b-kube-api-access-kxk6h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p784w\" (UID: \"81826a66-3411-449f-87fe-071f65d2165b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:15:58.788899 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.788755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb6723e3-baac-40a3-bfef-aa44a9f62d64-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bz69h\" (UID: \"eb6723e3-baac-40a3-bfef-aa44a9f62d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:15:58.789085 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.789053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81826a66-3411-449f-87fe-071f65d2165b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p784w\" (UID: \"81826a66-3411-449f-87fe-071f65d2165b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:15:58.789322 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.789305 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb6723e3-baac-40a3-bfef-aa44a9f62d64-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bz69h\" (UID: \"eb6723e3-baac-40a3-bfef-aa44a9f62d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:15:58.789556 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.789531 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81826a66-3411-449f-87fe-071f65d2165b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p784w\" (UID: \"81826a66-3411-449f-87fe-071f65d2165b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:15:58.802882 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.802853 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxk6h\" (UniqueName: \"kubernetes.io/projected/81826a66-3411-449f-87fe-071f65d2165b-kube-api-access-kxk6h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p784w\" (UID: \"81826a66-3411-449f-87fe-071f65d2165b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:15:58.803016 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.802900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2czl\" (UniqueName: \"kubernetes.io/projected/eb6723e3-baac-40a3-bfef-aa44a9f62d64-kube-api-access-h2czl\") pod \"kuadrant-operator-controller-manager-86878c769f-bz69h\" (UID: \"eb6723e3-baac-40a3-bfef-aa44a9f62d64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:15:58.900878 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.900796 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:15:58.921265 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:58.921235 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:15:59.129882 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:59.129845 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48c50310-46c0-4b48-b01c-e9a057993e88" path="/var/lib/kubelet/pods/48c50310-46c0-4b48-b01c-e9a057993e88/volumes" Jul 22 16:15:59.577921 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:15:59.577884 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:15:59.587151 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:15:59.587110 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-9bntr\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="48c50310-46c0-4b48-b01c-e9a057993e88" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9bntr" Jul 22 16:16:10.312563 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.312535 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:16:10.315024 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.314987 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rsvf6\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="639a1187-e723-40f9-8f8a-beb5d27889d6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:16:10.324212 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.324184 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:16:10.327113 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.326968 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:16:10.329350 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.329314 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rsvf6\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="639a1187-e723-40f9-8f8a-beb5d27889d6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:16:10.331647 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.331606 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h"] Jul 22 16:16:10.333068 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:16:10.333046 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb6723e3_baac_40a3_bfef_aa44a9f62d64.slice/crio-fe54549698bf32f3a79a66d6931baa2139c1eae95ac4e7e7fdd2766355155e26 WatchSource:0}: Error finding container fe54549698bf32f3a79a66d6931baa2139c1eae95ac4e7e7fdd2766355155e26: Status 404 returned error can't find the container with id fe54549698bf32f3a79a66d6931baa2139c1eae95ac4e7e7fdd2766355155e26 Jul 22 16:16:10.353738 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.353715 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w"] Jul 22 16:16:10.355363 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:16:10.355338 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81826a66_3411_449f_87fe_071f65d2165b.slice/crio-49b176deddda39bb9ec93393f26e3c1631f8150bcbcd07eac0cd75e460fe057f WatchSource:0}: Error finding container 49b176deddda39bb9ec93393f26e3c1631f8150bcbcd07eac0cd75e460fe057f: Status 404 returned error can't find the container with id 49b176deddda39bb9ec93393f26e3c1631f8150bcbcd07eac0cd75e460fe057f Jul 22 16:16:10.396518 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.396488 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlglz\" (UniqueName: \"kubernetes.io/projected/d133109e-430f-4439-9ce5-5a277ae059c6-kube-api-access-dlglz\") pod \"d133109e-430f-4439-9ce5-5a277ae059c6\" (UID: \"d133109e-430f-4439-9ce5-5a277ae059c6\") " Jul 22 16:16:10.396669 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.396568 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkg2m\" (UniqueName: \"kubernetes.io/projected/639a1187-e723-40f9-8f8a-beb5d27889d6-kube-api-access-dkg2m\") pod \"639a1187-e723-40f9-8f8a-beb5d27889d6\" (UID: \"639a1187-e723-40f9-8f8a-beb5d27889d6\") " Jul 22 16:16:10.396669 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.396616 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d133109e-430f-4439-9ce5-5a277ae059c6-extensions-socket-volume\") pod \"d133109e-430f-4439-9ce5-5a277ae059c6\" (UID: \"d133109e-430f-4439-9ce5-5a277ae059c6\") " Jul 22 16:16:10.397241 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.397206 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d133109e-430f-4439-9ce5-5a277ae059c6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d133109e-430f-4439-9ce5-5a277ae059c6" (UID: "d133109e-430f-4439-9ce5-5a277ae059c6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:16:10.398735 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.398706 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d133109e-430f-4439-9ce5-5a277ae059c6-kube-api-access-dlglz" (OuterVolumeSpecName: "kube-api-access-dlglz") pod "d133109e-430f-4439-9ce5-5a277ae059c6" (UID: "d133109e-430f-4439-9ce5-5a277ae059c6"). InnerVolumeSpecName "kube-api-access-dlglz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:16:10.398912 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.398890 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/639a1187-e723-40f9-8f8a-beb5d27889d6-kube-api-access-dkg2m" (OuterVolumeSpecName: "kube-api-access-dkg2m") pod "639a1187-e723-40f9-8f8a-beb5d27889d6" (UID: "639a1187-e723-40f9-8f8a-beb5d27889d6"). InnerVolumeSpecName "kube-api-access-dkg2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:16:10.497868 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.497830 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlglz\" (UniqueName: \"kubernetes.io/projected/d133109e-430f-4439-9ce5-5a277ae059c6-kube-api-access-dlglz\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:16:10.497868 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.497863 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkg2m\" (UniqueName: \"kubernetes.io/projected/639a1187-e723-40f9-8f8a-beb5d27889d6-kube-api-access-dkg2m\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:16:10.497868 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.497873 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d133109e-430f-4439-9ce5-5a277ae059c6-extensions-socket-volume\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:16:10.645694 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.645587 2564 generic.go:356] "Generic (PLEG): container finished" podID="639a1187-e723-40f9-8f8a-beb5d27889d6" containerID="a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408" exitCode=0 Jul 22 16:16:10.645694 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.645659 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:16:10.645912 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.645699 2564 scope.go:117] "RemoveContainer" containerID="a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408" Jul 22 16:16:10.647676 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.647651 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" event={"ID":"eb6723e3-baac-40a3-bfef-aa44a9f62d64","Type":"ContainerStarted","Data":"3a4bf4be117f54232706d06d5ee62c96089567606bdd48c860a373334aed759e"} Jul 22 16:16:10.647820 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.647739 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" event={"ID":"eb6723e3-baac-40a3-bfef-aa44a9f62d64","Type":"ContainerStarted","Data":"fe54549698bf32f3a79a66d6931baa2139c1eae95ac4e7e7fdd2766355155e26"} Jul 22 16:16:10.647898 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.647878 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:16:10.648592 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.648562 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rsvf6\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="639a1187-e723-40f9-8f8a-beb5d27889d6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:16:10.649170 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.649151 2564 generic.go:356] "Generic (PLEG): container finished" podID="d133109e-430f-4439-9ce5-5a277ae059c6" containerID="b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7" exitCode=0 Jul 22 16:16:10.649294 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.649210 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:16:10.651031 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.651003 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:16:10.651160 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.651086 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" event={"ID":"d0d4aeb7-5c31-4e66-b082-a6f2160e7456","Type":"ContainerStarted","Data":"3ead5b341a216f9e54b5ed7060cc164ade9d28def814e67eae2d4824823282ec"} Jul 22 16:16:10.652990 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.652964 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" event={"ID":"81826a66-3411-449f-87fe-071f65d2165b","Type":"ContainerStarted","Data":"7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b"} Jul 22 16:16:10.653097 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.653011 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" event={"ID":"81826a66-3411-449f-87fe-071f65d2165b","Type":"ContainerStarted","Data":"49b176deddda39bb9ec93393f26e3c1631f8150bcbcd07eac0cd75e460fe057f"} Jul 22 16:16:10.653164 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.653114 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:16:10.657020 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.657004 2564 scope.go:117] "RemoveContainer" containerID="a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408" Jul 22 16:16:10.657247 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.657226 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408\": container with ID starting with a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408 not found: ID does not exist" containerID="a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408" Jul 22 16:16:10.657322 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.657260 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408"} err="failed to get container status \"a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408\": rpc error: code = NotFound desc = could not find container \"a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408\": container with ID starting with a08fe658adbd448f4fc9aed60cd8cce833d55dfdb5ae2c279f991f1a9bc1d408 not found: ID does not exist" Jul 22 16:16:10.657322 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.657286 2564 scope.go:117] "RemoveContainer" containerID="b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7" Jul 22 16:16:10.666043 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.665923 2564 scope.go:117] "RemoveContainer" containerID="b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7" Jul 22 16:16:10.666214 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.666195 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7\": container with ID starting with b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7 not found: ID does not exist" containerID="b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7" Jul 22 16:16:10.666261 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.666221 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7"} err="failed to get container status \"b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7\": rpc error: code = NotFound desc = could not find container \"b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7\": container with ID starting with b4e2faab4d036e553c5efe202dd8a0f8e6dd8d05756e24cd7b14fbfcd52f6db7 not found: ID does not exist" Jul 22 16:16:10.674321 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.674267 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" podStartSLOduration=12.674251586 podStartE2EDuration="12.674251586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:15:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:16:10.671983159 +0000 UTC m=+534.067524995" watchObservedRunningTime="2026-07-22 16:16:10.674251586 +0000 UTC m=+534.069793421" Jul 22 16:16:10.674662 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.674618 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:16:10.676865 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.676840 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rsvf6\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="639a1187-e723-40f9-8f8a-beb5d27889d6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:16:10.678947 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.678923 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-j26td\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j26td" Jul 22 16:16:10.699336 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.699283 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nsd9z" podStartSLOduration=1.012709649 podStartE2EDuration="22.699266944s" totalImagesPullingTime="21.686557295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:16:10.69689863 +0000 UTC m=+534.092440465" watchObservedRunningTime="2026-07-22 16:16:10.699266944 +0000 UTC m=+534.094808778" Jul 22 16:16:10.700939 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:10.700914 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rsvf6\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="639a1187-e723-40f9-8f8a-beb5d27889d6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rsvf6" Jul 22 16:16:10.730719 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:10.730651 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" podStartSLOduration=12.730604041 podStartE2EDuration="12.730604041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:15:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:16:10.726494675 +0000 UTC m=+534.122036526" watchObservedRunningTime="2026-07-22 16:16:10.730604041 +0000 UTC m=+534.126145876" Jul 22 16:16:11.128891 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:11.128856 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="639a1187-e723-40f9-8f8a-beb5d27889d6" path="/var/lib/kubelet/pods/639a1187-e723-40f9-8f8a-beb5d27889d6/volumes" Jul 22 16:16:11.129212 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:11.129198 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d133109e-430f-4439-9ce5-5a277ae059c6" path="/var/lib/kubelet/pods/d133109e-430f-4439-9ce5-5a277ae059c6/volumes" Jul 22 16:16:21.664261 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:21.664227 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:16:21.664822 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:21.664310 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:16:21.739178 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:21.739135 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w"] Jul 22 16:16:21.739395 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:21.739368 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" podUID="81826a66-3411-449f-87fe-071f65d2165b" containerName="manager" containerID="cri-o://7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b" gracePeriod=10 Jul 22 16:16:21.990898 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:21.990872 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:16:22.107338 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.107302 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxk6h\" (UniqueName: \"kubernetes.io/projected/81826a66-3411-449f-87fe-071f65d2165b-kube-api-access-kxk6h\") pod \"81826a66-3411-449f-87fe-071f65d2165b\" (UID: \"81826a66-3411-449f-87fe-071f65d2165b\") " Jul 22 16:16:22.107338 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.107340 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81826a66-3411-449f-87fe-071f65d2165b-extensions-socket-volume\") pod \"81826a66-3411-449f-87fe-071f65d2165b\" (UID: \"81826a66-3411-449f-87fe-071f65d2165b\") " Jul 22 16:16:22.107809 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.107776 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81826a66-3411-449f-87fe-071f65d2165b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "81826a66-3411-449f-87fe-071f65d2165b" (UID: "81826a66-3411-449f-87fe-071f65d2165b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:16:22.109408 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.109388 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81826a66-3411-449f-87fe-071f65d2165b-kube-api-access-kxk6h" (OuterVolumeSpecName: "kube-api-access-kxk6h") pod "81826a66-3411-449f-87fe-071f65d2165b" (UID: "81826a66-3411-449f-87fe-071f65d2165b"). InnerVolumeSpecName "kube-api-access-kxk6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:16:22.208934 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.208854 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxk6h\" (UniqueName: \"kubernetes.io/projected/81826a66-3411-449f-87fe-071f65d2165b-kube-api-access-kxk6h\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:16:22.208934 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.208920 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81826a66-3411-449f-87fe-071f65d2165b-extensions-socket-volume\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:16:22.721396 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.721362 2564 generic.go:356] "Generic (PLEG): container finished" podID="81826a66-3411-449f-87fe-071f65d2165b" containerID="7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b" exitCode=0 Jul 22 16:16:22.721840 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.721408 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" Jul 22 16:16:22.721840 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.721447 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" event={"ID":"81826a66-3411-449f-87fe-071f65d2165b","Type":"ContainerDied","Data":"7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b"} Jul 22 16:16:22.721840 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.721504 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w" event={"ID":"81826a66-3411-449f-87fe-071f65d2165b","Type":"ContainerDied","Data":"49b176deddda39bb9ec93393f26e3c1631f8150bcbcd07eac0cd75e460fe057f"} Jul 22 16:16:22.721840 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.721533 2564 scope.go:117] "RemoveContainer" containerID="7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b" Jul 22 16:16:22.730984 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.730959 2564 scope.go:117] "RemoveContainer" containerID="7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b" Jul 22 16:16:22.731233 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:22.731214 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b\": container with ID starting with 7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b not found: ID does not exist" containerID="7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b" Jul 22 16:16:22.731281 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.731242 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b"} err="failed to get container status \"7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b\": rpc error: code = NotFound desc = could not find container \"7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b\": container with ID starting with 7fea342b3e815f25eb22d256707f29cd13806c713aa9fccc34c8def48b637b7b not found: ID does not exist" Jul 22 16:16:22.762363 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.762330 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w"] Jul 22 16:16:22.765961 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:22.765938 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p784w"] Jul 22 16:16:23.129015 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:23.128935 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81826a66-3411-449f-87fe-071f65d2165b" path="/var/lib/kubelet/pods/81826a66-3411-449f-87fe-071f65d2165b/volumes" Jul 22 16:16:38.114782 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.114746 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf"] Jul 22 16:16:38.115487 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.115468 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81826a66-3411-449f-87fe-071f65d2165b" containerName="manager" Jul 22 16:16:38.115592 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.115491 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="81826a66-3411-449f-87fe-071f65d2165b" containerName="manager" Jul 22 16:16:38.115592 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.115532 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639a1187-e723-40f9-8f8a-beb5d27889d6" containerName="manager" Jul 22 16:16:38.115592 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.115540 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="639a1187-e723-40f9-8f8a-beb5d27889d6" containerName="manager" Jul 22 16:16:38.115801 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.115653 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="81826a66-3411-449f-87fe-071f65d2165b" containerName="manager" Jul 22 16:16:38.115801 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.115673 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="639a1187-e723-40f9-8f8a-beb5d27889d6" containerName="manager" Jul 22 16:16:38.367917 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.367816 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf"] Jul 22 16:16:38.367917 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.367842 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.370678 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.370647 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 16:16:38.370678 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.370672 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tbstr\"" Jul 22 16:16:38.427873 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.427831 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.428066 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.427886 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.428066 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.427911 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.428066 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.427949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.428066 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.428029 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.428249 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.428075 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.428249 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.428093 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74c9h\" (UniqueName: \"kubernetes.io/projected/3127e95b-74b6-4ef2-ace4-63bba9953ffc-kube-api-access-74c9h\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.428249 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.428116 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.428249 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.428155 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.528756 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.528713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.528756 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.528754 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529037 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.528776 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74c9h\" (UniqueName: \"kubernetes.io/projected/3127e95b-74b6-4ef2-ace4-63bba9953ffc-kube-api-access-74c9h\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529037 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.528796 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529037 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.528834 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529037 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.528861 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529037 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.528910 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529037 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.529004 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529328 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.529051 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529401 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.529373 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529533 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.529450 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529533 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.529514 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529533 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.529524 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.529739 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.529726 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.531344 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.531322 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.531506 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.531489 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.538289 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.538266 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3127e95b-74b6-4ef2-ace4-63bba9953ffc-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.538521 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.538495 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74c9h\" (UniqueName: \"kubernetes.io/projected/3127e95b-74b6-4ef2-ace4-63bba9953ffc-kube-api-access-74c9h\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fwbxf\" (UID: \"3127e95b-74b6-4ef2-ace4-63bba9953ffc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.678919 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.678814 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:38.830544 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:38.830519 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf"] Jul 22 16:16:38.832564 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:16:38.832526 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3127e95b_74b6_4ef2_ace4_63bba9953ffc.slice/crio-8e641073c64fcb52c3bf910a80087dadc057ddbc62ed46c7120536dc0174cc8f WatchSource:0}: Error finding container 8e641073c64fcb52c3bf910a80087dadc057ddbc62ed46c7120536dc0174cc8f: Status 404 returned error can't find the container with id 8e641073c64fcb52c3bf910a80087dadc057ddbc62ed46c7120536dc0174cc8f Jul 22 16:16:39.821990 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:39.821941 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" event={"ID":"3127e95b-74b6-4ef2-ace4-63bba9953ffc","Type":"ContainerStarted","Data":"8e641073c64fcb52c3bf910a80087dadc057ddbc62ed46c7120536dc0174cc8f"} Jul 22 16:16:41.654842 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:41.654809 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 22 16:16:41.655130 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:41.654884 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 22 16:16:41.655130 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:41.654915 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 22 16:16:41.836963 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:41.836923 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" event={"ID":"3127e95b-74b6-4ef2-ace4-63bba9953ffc","Type":"ContainerStarted","Data":"eb26587767537242f30d60e3982bdff8f5b61bf292657a880f2243c36357a921"} Jul 22 16:16:41.861648 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:41.861576 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" podStartSLOduration=1.04194119 podStartE2EDuration="3.861559441s" totalImagesPullingTime="2.819618251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:16:41.859819604 +0000 UTC m=+565.255361449" watchObservedRunningTime="2026-07-22 16:16:41.861559441 +0000 UTC m=+565.257101275" Jul 22 16:16:42.680057 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:42.680013 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:42.684993 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:42.684968 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:42.843407 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:42.843369 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:42.844613 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:42.844591 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:16:42.927768 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:42.927734 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rdt9b"] Jul 22 16:16:42.930860 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:42.930814 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:42.933826 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:42.933792 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 22 16:16:42.942817 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:42.942796 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rdt9b"] Jul 22 16:16:42.967068 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:42.967030 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdqrh\" (UniqueName: \"kubernetes.io/projected/e4706001-f1a9-4795-981e-e1deb2027981-kube-api-access-zdqrh\") pod \"limitador-limitador-64c94fd64f-rdt9b\" (UID: \"e4706001-f1a9-4795-981e-e1deb2027981\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:42.967240 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:42.967077 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e4706001-f1a9-4795-981e-e1deb2027981-config-file\") pod \"limitador-limitador-64c94fd64f-rdt9b\" (UID: \"e4706001-f1a9-4795-981e-e1deb2027981\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:43.020499 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.020455 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rdt9b"] Jul 22 16:16:43.068259 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.068226 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdqrh\" (UniqueName: \"kubernetes.io/projected/e4706001-f1a9-4795-981e-e1deb2027981-kube-api-access-zdqrh\") pod \"limitador-limitador-64c94fd64f-rdt9b\" (UID: \"e4706001-f1a9-4795-981e-e1deb2027981\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:43.068440 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.068275 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e4706001-f1a9-4795-981e-e1deb2027981-config-file\") pod \"limitador-limitador-64c94fd64f-rdt9b\" (UID: \"e4706001-f1a9-4795-981e-e1deb2027981\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:43.068979 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.068957 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e4706001-f1a9-4795-981e-e1deb2027981-config-file\") pod \"limitador-limitador-64c94fd64f-rdt9b\" (UID: \"e4706001-f1a9-4795-981e-e1deb2027981\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:43.077470 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.077445 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdqrh\" (UniqueName: \"kubernetes.io/projected/e4706001-f1a9-4795-981e-e1deb2027981-kube-api-access-zdqrh\") pod \"limitador-limitador-64c94fd64f-rdt9b\" (UID: \"e4706001-f1a9-4795-981e-e1deb2027981\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:43.241908 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.241873 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:43.386499 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.386472 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rdt9b"] Jul 22 16:16:43.388251 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:16:43.388201 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4706001_f1a9_4795_981e_e1deb2027981.slice/crio-bdc05c4f7e0b5be2609c2bed25e76777f1b346193c819956766ada6df6276396 WatchSource:0}: Error finding container bdc05c4f7e0b5be2609c2bed25e76777f1b346193c819956766ada6df6276396: Status 404 returned error can't find the container with id bdc05c4f7e0b5be2609c2bed25e76777f1b346193c819956766ada6df6276396 Jul 22 16:16:43.781427 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.781389 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-4j6vr"] Jul 22 16:16:43.786115 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.786095 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4j6vr" Jul 22 16:16:43.788989 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.788949 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gn4s2\"" Jul 22 16:16:43.792270 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.792245 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4j6vr"] Jul 22 16:16:43.850133 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.850100 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" event={"ID":"e4706001-f1a9-4795-981e-e1deb2027981","Type":"ContainerStarted","Data":"bdc05c4f7e0b5be2609c2bed25e76777f1b346193c819956766ada6df6276396"} Jul 22 16:16:43.876009 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.875977 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpvc2\" (UniqueName: \"kubernetes.io/projected/706030e5-4887-432b-980a-72da17e530c3-kube-api-access-wpvc2\") pod \"authorino-58cd955675-4j6vr\" (UID: \"706030e5-4887-432b-980a-72da17e530c3\") " pod="kuadrant-system/authorino-58cd955675-4j6vr" Jul 22 16:16:43.976916 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.976874 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpvc2\" (UniqueName: \"kubernetes.io/projected/706030e5-4887-432b-980a-72da17e530c3-kube-api-access-wpvc2\") pod \"authorino-58cd955675-4j6vr\" (UID: \"706030e5-4887-432b-980a-72da17e530c3\") " pod="kuadrant-system/authorino-58cd955675-4j6vr" Jul 22 16:16:43.990735 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:43.990707 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpvc2\" (UniqueName: \"kubernetes.io/projected/706030e5-4887-432b-980a-72da17e530c3-kube-api-access-wpvc2\") pod \"authorino-58cd955675-4j6vr\" (UID: \"706030e5-4887-432b-980a-72da17e530c3\") " pod="kuadrant-system/authorino-58cd955675-4j6vr" Jul 22 16:16:44.097635 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:44.097534 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4j6vr" Jul 22 16:16:44.263660 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:44.263455 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-4j6vr"] Jul 22 16:16:44.266381 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:16:44.266345 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod706030e5_4887_432b_980a_72da17e530c3.slice/crio-a61adfe194aeef674d8463d662e919e531274a457ad7abaa7911475b83a0b2a6 WatchSource:0}: Error finding container a61adfe194aeef674d8463d662e919e531274a457ad7abaa7911475b83a0b2a6: Status 404 returned error can't find the container with id a61adfe194aeef674d8463d662e919e531274a457ad7abaa7911475b83a0b2a6 Jul 22 16:16:44.857988 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:44.857949 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4j6vr" event={"ID":"706030e5-4887-432b-980a-72da17e530c3","Type":"ContainerStarted","Data":"a61adfe194aeef674d8463d662e919e531274a457ad7abaa7911475b83a0b2a6"} Jul 22 16:16:46.876169 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:46.876123 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" event={"ID":"e4706001-f1a9-4795-981e-e1deb2027981","Type":"ContainerStarted","Data":"31d7a9a3120361452d40462901a98a3f3e45291b053ff7913fccb4d5f8459826"} Jul 22 16:16:46.876812 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:46.876322 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:46.897506 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:46.897438 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" podStartSLOduration=1.915670744 podStartE2EDuration="4.897419406s" totalImagesPullingTime="2.981748662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:16:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:16:46.896333339 +0000 UTC m=+570.291875189" watchObservedRunningTime="2026-07-22 16:16:46.897419406 +0000 UTC m=+570.292961261" Jul 22 16:16:48.845113 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:48.845079 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4j6vr"] Jul 22 16:16:48.889262 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:48.889223 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4j6vr" event={"ID":"706030e5-4887-432b-980a-72da17e530c3","Type":"ContainerStarted","Data":"5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d"} Jul 22 16:16:48.909616 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:48.909553 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-4j6vr" podStartSLOduration=1.712408891 podStartE2EDuration="5.909536312s" totalImagesPullingTime="4.197127421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:16:48.908608319 +0000 UTC m=+572.304150154" watchObservedRunningTime="2026-07-22 16:16:48.909536312 +0000 UTC m=+572.305078145" Jul 22 16:16:49.894919 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:49.894877 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-4j6vr" podUID="706030e5-4887-432b-980a-72da17e530c3" containerName="authorino" containerID="cri-o://5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d" gracePeriod=30 Jul 22 16:16:50.148568 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.148488 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4j6vr" Jul 22 16:16:50.239961 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.239927 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpvc2\" (UniqueName: \"kubernetes.io/projected/706030e5-4887-432b-980a-72da17e530c3-kube-api-access-wpvc2\") pod \"706030e5-4887-432b-980a-72da17e530c3\" (UID: \"706030e5-4887-432b-980a-72da17e530c3\") " Jul 22 16:16:50.242033 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.242003 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/706030e5-4887-432b-980a-72da17e530c3-kube-api-access-wpvc2" (OuterVolumeSpecName: "kube-api-access-wpvc2") pod "706030e5-4887-432b-980a-72da17e530c3" (UID: "706030e5-4887-432b-980a-72da17e530c3"). InnerVolumeSpecName "kube-api-access-wpvc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:16:50.341440 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.341396 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpvc2\" (UniqueName: \"kubernetes.io/projected/706030e5-4887-432b-980a-72da17e530c3-kube-api-access-wpvc2\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:16:50.902458 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.902421 2564 generic.go:356] "Generic (PLEG): container finished" podID="706030e5-4887-432b-980a-72da17e530c3" containerID="5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d" exitCode=0 Jul 22 16:16:50.902918 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.902480 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-4j6vr" Jul 22 16:16:50.902918 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.902489 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4j6vr" event={"ID":"706030e5-4887-432b-980a-72da17e530c3","Type":"ContainerDied","Data":"5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d"} Jul 22 16:16:50.902918 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.902531 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-4j6vr" event={"ID":"706030e5-4887-432b-980a-72da17e530c3","Type":"ContainerDied","Data":"a61adfe194aeef674d8463d662e919e531274a457ad7abaa7911475b83a0b2a6"} Jul 22 16:16:50.902918 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.902550 2564 scope.go:117] "RemoveContainer" containerID="5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d" Jul 22 16:16:50.912496 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.912476 2564 scope.go:117] "RemoveContainer" containerID="5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d" Jul 22 16:16:50.912801 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:16:50.912783 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d\": container with ID starting with 5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d not found: ID does not exist" containerID="5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d" Jul 22 16:16:50.912879 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.912809 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d"} err="failed to get container status \"5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d\": rpc error: code = NotFound desc = could not find container \"5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d\": container with ID starting with 5e8f09766b5542569a4835d24c9b92cc80c27c2e8d9edff46e4ddea99634b66d not found: ID does not exist" Jul 22 16:16:50.926048 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.926019 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-4j6vr"] Jul 22 16:16:50.930807 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:50.930782 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-4j6vr"] Jul 22 16:16:51.128897 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:51.128857 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="706030e5-4887-432b-980a-72da17e530c3" path="/var/lib/kubelet/pods/706030e5-4887-432b-980a-72da17e530c3/volumes" Jul 22 16:16:57.490984 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:57.490945 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rdt9b"] Jul 22 16:16:57.491485 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:57.491180 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" podUID="e4706001-f1a9-4795-981e-e1deb2027981" containerName="limitador" containerID="cri-o://31d7a9a3120361452d40462901a98a3f3e45291b053ff7913fccb4d5f8459826" gracePeriod=30 Jul 22 16:16:57.492312 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:57.492289 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:57.943152 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:57.943113 2564 generic.go:356] "Generic (PLEG): container finished" podID="e4706001-f1a9-4795-981e-e1deb2027981" containerID="31d7a9a3120361452d40462901a98a3f3e45291b053ff7913fccb4d5f8459826" exitCode=0 Jul 22 16:16:57.943334 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:57.943185 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" event={"ID":"e4706001-f1a9-4795-981e-e1deb2027981","Type":"ContainerDied","Data":"31d7a9a3120361452d40462901a98a3f3e45291b053ff7913fccb4d5f8459826"} Jul 22 16:16:58.071420 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.071391 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:58.223810 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.223774 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdqrh\" (UniqueName: \"kubernetes.io/projected/e4706001-f1a9-4795-981e-e1deb2027981-kube-api-access-zdqrh\") pod \"e4706001-f1a9-4795-981e-e1deb2027981\" (UID: \"e4706001-f1a9-4795-981e-e1deb2027981\") " Jul 22 16:16:58.224025 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.223889 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e4706001-f1a9-4795-981e-e1deb2027981-config-file\") pod \"e4706001-f1a9-4795-981e-e1deb2027981\" (UID: \"e4706001-f1a9-4795-981e-e1deb2027981\") " Jul 22 16:16:58.224243 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.224214 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4706001-f1a9-4795-981e-e1deb2027981-config-file" (OuterVolumeSpecName: "config-file") pod "e4706001-f1a9-4795-981e-e1deb2027981" (UID: "e4706001-f1a9-4795-981e-e1deb2027981"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:16:58.225961 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.225944 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4706001-f1a9-4795-981e-e1deb2027981-kube-api-access-zdqrh" (OuterVolumeSpecName: "kube-api-access-zdqrh") pod "e4706001-f1a9-4795-981e-e1deb2027981" (UID: "e4706001-f1a9-4795-981e-e1deb2027981"). InnerVolumeSpecName "kube-api-access-zdqrh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:16:58.325496 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.325401 2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e4706001-f1a9-4795-981e-e1deb2027981-config-file\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:16:58.325496 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.325437 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdqrh\" (UniqueName: \"kubernetes.io/projected/e4706001-f1a9-4795-981e-e1deb2027981-kube-api-access-zdqrh\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:16:58.953715 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.953685 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" Jul 22 16:16:58.953715 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.953701 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rdt9b" event={"ID":"e4706001-f1a9-4795-981e-e1deb2027981","Type":"ContainerDied","Data":"bdc05c4f7e0b5be2609c2bed25e76777f1b346193c819956766ada6df6276396"} Jul 22 16:16:58.954187 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.953764 2564 scope.go:117] "RemoveContainer" containerID="31d7a9a3120361452d40462901a98a3f3e45291b053ff7913fccb4d5f8459826" Jul 22 16:16:58.976766 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.976739 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rdt9b"] Jul 22 16:16:58.980611 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:58.980584 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rdt9b"] Jul 22 16:16:59.128919 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.128885 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4706001-f1a9-4795-981e-e1deb2027981" path="/var/lib/kubelet/pods/e4706001-f1a9-4795-981e-e1deb2027981/volumes" Jul 22 16:16:59.600778 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.600742 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh"] Jul 22 16:16:59.601318 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.601295 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4706001-f1a9-4795-981e-e1deb2027981" containerName="limitador" Jul 22 16:16:59.601510 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.601498 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4706001-f1a9-4795-981e-e1deb2027981" containerName="limitador" Jul 22 16:16:59.601618 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.601607 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706030e5-4887-432b-980a-72da17e530c3" containerName="authorino" Jul 22 16:16:59.601721 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.601711 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="706030e5-4887-432b-980a-72da17e530c3" containerName="authorino" Jul 22 16:16:59.601917 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.601903 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="706030e5-4887-432b-980a-72da17e530c3" containerName="authorino" Jul 22 16:16:59.602012 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.602003 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4706001-f1a9-4795-981e-e1deb2027981" containerName="limitador" Jul 22 16:16:59.607004 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.606982 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" Jul 22 16:16:59.614575 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.614533 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 16:16:59.615010 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.614907 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 16:16:59.615010 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.614939 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cmq5j\"" Jul 22 16:16:59.615453 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.615431 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 22 16:16:59.626402 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.626373 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh"] Jul 22 16:16:59.738174 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.738137 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s96xk\" (UniqueName: \"kubernetes.io/projected/57ce2cf3-cbbb-4494-b240-cf951620c0c6-kube-api-access-s96xk\") pod \"postgres-69fb4594cf-9qpgh\" (UID: \"57ce2cf3-cbbb-4494-b240-cf951620c0c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" Jul 22 16:16:59.738174 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.738177 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/57ce2cf3-cbbb-4494-b240-cf951620c0c6-data\") pod \"postgres-69fb4594cf-9qpgh\" (UID: \"57ce2cf3-cbbb-4494-b240-cf951620c0c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" Jul 22 16:16:59.839281 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.839241 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s96xk\" (UniqueName: \"kubernetes.io/projected/57ce2cf3-cbbb-4494-b240-cf951620c0c6-kube-api-access-s96xk\") pod \"postgres-69fb4594cf-9qpgh\" (UID: \"57ce2cf3-cbbb-4494-b240-cf951620c0c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" Jul 22 16:16:59.839281 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.839283 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/57ce2cf3-cbbb-4494-b240-cf951620c0c6-data\") pod \"postgres-69fb4594cf-9qpgh\" (UID: \"57ce2cf3-cbbb-4494-b240-cf951620c0c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" Jul 22 16:16:59.839720 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.839700 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/57ce2cf3-cbbb-4494-b240-cf951620c0c6-data\") pod \"postgres-69fb4594cf-9qpgh\" (UID: \"57ce2cf3-cbbb-4494-b240-cf951620c0c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" Jul 22 16:16:59.851925 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.851858 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s96xk\" (UniqueName: \"kubernetes.io/projected/57ce2cf3-cbbb-4494-b240-cf951620c0c6-kube-api-access-s96xk\") pod \"postgres-69fb4594cf-9qpgh\" (UID: \"57ce2cf3-cbbb-4494-b240-cf951620c0c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" Jul 22 16:16:59.919214 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:16:59.919177 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" Jul 22 16:17:00.068016 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:00.067979 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh"] Jul 22 16:17:00.071573 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:17:00.071542 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ce2cf3_cbbb_4494_b240_cf951620c0c6.slice/crio-9d3d9a9ccb1cb070b2c1929a4d9bd11963ed51138a4ffa9c5a1f8895960cef32 WatchSource:0}: Error finding container 9d3d9a9ccb1cb070b2c1929a4d9bd11963ed51138a4ffa9c5a1f8895960cef32: Status 404 returned error can't find the container with id 9d3d9a9ccb1cb070b2c1929a4d9bd11963ed51138a4ffa9c5a1f8895960cef32 Jul 22 16:17:00.970331 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:00.970288 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" event={"ID":"57ce2cf3-cbbb-4494-b240-cf951620c0c6","Type":"ContainerStarted","Data":"9d3d9a9ccb1cb070b2c1929a4d9bd11963ed51138a4ffa9c5a1f8895960cef32"} Jul 22 16:17:06.003556 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:06.003515 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" event={"ID":"57ce2cf3-cbbb-4494-b240-cf951620c0c6","Type":"ContainerStarted","Data":"b465e4806a019dca08d5f6de51b10ce3da0c390cb6d76c10e62f8cebafe16ec0"} Jul 22 16:17:06.004084 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:06.003685 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" Jul 22 16:17:06.024157 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:06.024105 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" podStartSLOduration=1.9917034999999998 podStartE2EDuration="7.024088471s" totalImagesPullingTime="5.032384971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:17:06.021411372 +0000 UTC m=+589.416953234" watchObservedRunningTime="2026-07-22 16:17:06.024088471 +0000 UTC m=+589.419630371" Jul 22 16:17:12.039362 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:12.039332 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9qpgh" Jul 22 16:17:20.041809 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.041759 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-658c7896cf-5l4wm"] Jul 22 16:17:20.147441 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.147407 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658c7896cf-5l4wm"] Jul 22 16:17:20.147638 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.147518 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:20.151655 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.151594 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6xqzg\"" Jul 22 16:17:20.151817 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.151666 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 16:17:20.152372 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.152338 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 16:17:20.223556 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.223518 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r996g\" (UniqueName: \"kubernetes.io/projected/8b9e6784-255c-4cbf-9954-7f478671fa9b-kube-api-access-r996g\") pod \"maas-controller-658c7896cf-5l4wm\" (UID: \"8b9e6784-255c-4cbf-9954-7f478671fa9b\") " pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:20.223556 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.223572 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b9e6784-255c-4cbf-9954-7f478671fa9b-webhook-cert\") pod \"maas-controller-658c7896cf-5l4wm\" (UID: \"8b9e6784-255c-4cbf-9954-7f478671fa9b\") " pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:20.240816 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.240781 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6848f74968-d46xl"] Jul 22 16:17:20.243679 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.243659 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:17:20.256101 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.256073 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6848f74968-d46xl"] Jul 22 16:17:20.324404 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.324299 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e581dbea-b9c5-4cca-875d-e2759f57cc46-webhook-cert\") pod \"maas-controller-6848f74968-d46xl\" (UID: \"e581dbea-b9c5-4cca-875d-e2759f57cc46\") " pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:17:20.324404 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.324373 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r996g\" (UniqueName: \"kubernetes.io/projected/8b9e6784-255c-4cbf-9954-7f478671fa9b-kube-api-access-r996g\") pod \"maas-controller-658c7896cf-5l4wm\" (UID: \"8b9e6784-255c-4cbf-9954-7f478671fa9b\") " pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:20.324681 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.324406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b9e6784-255c-4cbf-9954-7f478671fa9b-webhook-cert\") pod \"maas-controller-658c7896cf-5l4wm\" (UID: \"8b9e6784-255c-4cbf-9954-7f478671fa9b\") " pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:20.324681 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.324458 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2b2c\" (UniqueName: \"kubernetes.io/projected/e581dbea-b9c5-4cca-875d-e2759f57cc46-kube-api-access-n2b2c\") pod \"maas-controller-6848f74968-d46xl\" (UID: \"e581dbea-b9c5-4cca-875d-e2759f57cc46\") " pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:17:20.326958 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.326933 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b9e6784-255c-4cbf-9954-7f478671fa9b-webhook-cert\") pod \"maas-controller-658c7896cf-5l4wm\" (UID: \"8b9e6784-255c-4cbf-9954-7f478671fa9b\") " pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:20.334057 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.334030 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r996g\" (UniqueName: \"kubernetes.io/projected/8b9e6784-255c-4cbf-9954-7f478671fa9b-kube-api-access-r996g\") pod \"maas-controller-658c7896cf-5l4wm\" (UID: \"8b9e6784-255c-4cbf-9954-7f478671fa9b\") " pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:20.424990 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.424945 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e581dbea-b9c5-4cca-875d-e2759f57cc46-webhook-cert\") pod \"maas-controller-6848f74968-d46xl\" (UID: \"e581dbea-b9c5-4cca-875d-e2759f57cc46\") " pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:17:20.425198 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.425018 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2b2c\" (UniqueName: \"kubernetes.io/projected/e581dbea-b9c5-4cca-875d-e2759f57cc46-kube-api-access-n2b2c\") pod \"maas-controller-6848f74968-d46xl\" (UID: \"e581dbea-b9c5-4cca-875d-e2759f57cc46\") " pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:17:20.427491 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.427464 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e581dbea-b9c5-4cca-875d-e2759f57cc46-webhook-cert\") pod \"maas-controller-6848f74968-d46xl\" (UID: \"e581dbea-b9c5-4cca-875d-e2759f57cc46\") " pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:17:20.434867 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.434836 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2b2c\" (UniqueName: \"kubernetes.io/projected/e581dbea-b9c5-4cca-875d-e2759f57cc46-kube-api-access-n2b2c\") pod \"maas-controller-6848f74968-d46xl\" (UID: \"e581dbea-b9c5-4cca-875d-e2759f57cc46\") " pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:17:20.459365 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.459339 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:20.554019 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.553989 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:17:20.603824 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.603793 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-658c7896cf-5l4wm"] Jul 22 16:17:20.604216 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:17:20.604194 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9e6784_255c_4cbf_9954_7f478671fa9b.slice/crio-f79cf0827ca9496108990aca35d66d26c310f4d0dd01ea1ec68a850c7b983562 WatchSource:0}: Error finding container f79cf0827ca9496108990aca35d66d26c310f4d0dd01ea1ec68a850c7b983562: Status 404 returned error can't find the container with id f79cf0827ca9496108990aca35d66d26c310f4d0dd01ea1ec68a850c7b983562 Jul 22 16:17:20.703166 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:20.703133 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6848f74968-d46xl"] Jul 22 16:17:20.704364 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:17:20.704333 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode581dbea_b9c5_4cca_875d_e2759f57cc46.slice/crio-95f8a6f3a537e57888701edbe1368187bacc6a84ecf02e034e000d2a8f57a4b4 WatchSource:0}: Error finding container 95f8a6f3a537e57888701edbe1368187bacc6a84ecf02e034e000d2a8f57a4b4: Status 404 returned error can't find the container with id 95f8a6f3a537e57888701edbe1368187bacc6a84ecf02e034e000d2a8f57a4b4 Jul 22 16:17:21.085454 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:21.085415 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6848f74968-d46xl" event={"ID":"e581dbea-b9c5-4cca-875d-e2759f57cc46","Type":"ContainerStarted","Data":"95f8a6f3a537e57888701edbe1368187bacc6a84ecf02e034e000d2a8f57a4b4"} Jul 22 16:17:21.086816 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:21.086788 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658c7896cf-5l4wm" event={"ID":"8b9e6784-255c-4cbf-9954-7f478671fa9b","Type":"ContainerStarted","Data":"f79cf0827ca9496108990aca35d66d26c310f4d0dd01ea1ec68a850c7b983562"} Jul 22 16:17:24.105791 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:24.105750 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6848f74968-d46xl" event={"ID":"e581dbea-b9c5-4cca-875d-e2759f57cc46","Type":"ContainerStarted","Data":"daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96"} Jul 22 16:17:24.106270 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:24.105825 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:17:24.107603 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:24.107571 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658c7896cf-5l4wm" event={"ID":"8b9e6784-255c-4cbf-9954-7f478671fa9b","Type":"ContainerStarted","Data":"e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe"} Jul 22 16:17:24.107758 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:24.107674 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:24.149296 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:24.149243 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6848f74968-d46xl" podStartSLOduration=1.788097139 podStartE2EDuration="4.149224671s" totalImagesPullingTime="2.361127532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:17:24.14745905 +0000 UTC m=+607.543000884" watchObservedRunningTime="2026-07-22 16:17:24.149224671 +0000 UTC m=+607.544766505" Jul 22 16:17:24.179845 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:24.179780 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-658c7896cf-5l4wm" podStartSLOduration=1.712094165 podStartE2EDuration="4.179755714s" totalImagesPullingTime="2.467661549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:17:24.176161204 +0000 UTC m=+607.571703039" watchObservedRunningTime="2026-07-22 16:17:24.179755714 +0000 UTC m=+607.575297549" Jul 22 16:17:35.115446 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.115409 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:35.117073 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.115495 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:17:35.181928 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.181889 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-658c7896cf-5l4wm"] Jul 22 16:17:35.182163 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.182121 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-658c7896cf-5l4wm" podUID="8b9e6784-255c-4cbf-9954-7f478671fa9b" containerName="manager" containerID="cri-o://e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe" gracePeriod=30 Jul 22 16:17:35.435614 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.435580 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:35.559470 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.559433 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b9e6784-255c-4cbf-9954-7f478671fa9b-webhook-cert\") pod \"8b9e6784-255c-4cbf-9954-7f478671fa9b\" (UID: \"8b9e6784-255c-4cbf-9954-7f478671fa9b\") " Jul 22 16:17:35.559715 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.559500 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r996g\" (UniqueName: \"kubernetes.io/projected/8b9e6784-255c-4cbf-9954-7f478671fa9b-kube-api-access-r996g\") pod \"8b9e6784-255c-4cbf-9954-7f478671fa9b\" (UID: \"8b9e6784-255c-4cbf-9954-7f478671fa9b\") " Jul 22 16:17:35.561770 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.561741 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b9e6784-255c-4cbf-9954-7f478671fa9b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8b9e6784-255c-4cbf-9954-7f478671fa9b" (UID: "8b9e6784-255c-4cbf-9954-7f478671fa9b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:17:35.561770 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.561759 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9e6784-255c-4cbf-9954-7f478671fa9b-kube-api-access-r996g" (OuterVolumeSpecName: "kube-api-access-r996g") pod "8b9e6784-255c-4cbf-9954-7f478671fa9b" (UID: "8b9e6784-255c-4cbf-9954-7f478671fa9b"). InnerVolumeSpecName "kube-api-access-r996g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:17:35.660736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.660699 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b9e6784-255c-4cbf-9954-7f478671fa9b-webhook-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:17:35.660736 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:35.660729 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r996g\" (UniqueName: \"kubernetes.io/projected/8b9e6784-255c-4cbf-9954-7f478671fa9b-kube-api-access-r996g\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:17:36.174429 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:36.174387 2564 generic.go:356] "Generic (PLEG): container finished" podID="8b9e6784-255c-4cbf-9954-7f478671fa9b" containerID="e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe" exitCode=0 Jul 22 16:17:36.174903 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:36.174437 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-658c7896cf-5l4wm" Jul 22 16:17:36.174903 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:36.174477 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658c7896cf-5l4wm" event={"ID":"8b9e6784-255c-4cbf-9954-7f478671fa9b","Type":"ContainerDied","Data":"e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe"} Jul 22 16:17:36.174903 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:36.174531 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-658c7896cf-5l4wm" event={"ID":"8b9e6784-255c-4cbf-9954-7f478671fa9b","Type":"ContainerDied","Data":"f79cf0827ca9496108990aca35d66d26c310f4d0dd01ea1ec68a850c7b983562"} Jul 22 16:17:36.174903 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:36.174565 2564 scope.go:117] "RemoveContainer" containerID="e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe" Jul 22 16:17:36.184707 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:36.184488 2564 scope.go:117] "RemoveContainer" containerID="e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe" Jul 22 16:17:36.184839 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:17:36.184819 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe\": container with ID starting with e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe not found: ID does not exist" containerID="e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe" Jul 22 16:17:36.184884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:36.184847 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe"} err="failed to get container status \"e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe\": rpc error: code = NotFound desc = could not find container \"e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe\": container with ID starting with e615d80052e4a1cbe7fe5f6f38679e9a975140628f652a8b0c2972a5531eddfe not found: ID does not exist" Jul 22 16:17:36.199425 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:36.199401 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-658c7896cf-5l4wm"] Jul 22 16:17:36.205537 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:36.205513 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-658c7896cf-5l4wm"] Jul 22 16:17:37.129992 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:37.129961 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b9e6784-255c-4cbf-9954-7f478671fa9b" path="/var/lib/kubelet/pods/8b9e6784-255c-4cbf-9954-7f478671fa9b/volumes" Jul 22 16:17:55.672340 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.672305 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-56b457c467-xptbm"] Jul 22 16:17:55.672870 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.672732 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9e6784-255c-4cbf-9954-7f478671fa9b" containerName="manager" Jul 22 16:17:55.672870 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.672745 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9e6784-255c-4cbf-9954-7f478671fa9b" containerName="manager" Jul 22 16:17:55.672870 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.672805 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9e6784-255c-4cbf-9954-7f478671fa9b" containerName="manager" Jul 22 16:17:55.678166 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.678143 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:55.683074 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.683050 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 22 16:17:55.683199 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.683056 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vghv2\"" Jul 22 16:17:55.725696 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.725661 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-56b457c467-xptbm"] Jul 22 16:17:55.851416 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.851381 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lbht\" (UniqueName: \"kubernetes.io/projected/bcdb6efe-39a8-40f4-9630-478e51ada5b9-kube-api-access-9lbht\") pod \"maas-api-56b457c467-xptbm\" (UID: \"bcdb6efe-39a8-40f4-9630-478e51ada5b9\") " pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:55.851604 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.851440 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcdb6efe-39a8-40f4-9630-478e51ada5b9-openshift-service-ca\") pod \"maas-api-56b457c467-xptbm\" (UID: \"bcdb6efe-39a8-40f4-9630-478e51ada5b9\") " pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:55.851604 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.851519 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcdb6efe-39a8-40f4-9630-478e51ada5b9-maas-api-tls\") pod \"maas-api-56b457c467-xptbm\" (UID: \"bcdb6efe-39a8-40f4-9630-478e51ada5b9\") " pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:55.952588 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.952500 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcdb6efe-39a8-40f4-9630-478e51ada5b9-maas-api-tls\") pod \"maas-api-56b457c467-xptbm\" (UID: \"bcdb6efe-39a8-40f4-9630-478e51ada5b9\") " pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:55.952588 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.952566 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lbht\" (UniqueName: \"kubernetes.io/projected/bcdb6efe-39a8-40f4-9630-478e51ada5b9-kube-api-access-9lbht\") pod \"maas-api-56b457c467-xptbm\" (UID: \"bcdb6efe-39a8-40f4-9630-478e51ada5b9\") " pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:55.952863 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.952611 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcdb6efe-39a8-40f4-9630-478e51ada5b9-openshift-service-ca\") pod \"maas-api-56b457c467-xptbm\" (UID: \"bcdb6efe-39a8-40f4-9630-478e51ada5b9\") " pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:55.953218 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.953199 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcdb6efe-39a8-40f4-9630-478e51ada5b9-openshift-service-ca\") pod \"maas-api-56b457c467-xptbm\" (UID: \"bcdb6efe-39a8-40f4-9630-478e51ada5b9\") " pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:55.954950 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.954925 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcdb6efe-39a8-40f4-9630-478e51ada5b9-maas-api-tls\") pod \"maas-api-56b457c467-xptbm\" (UID: \"bcdb6efe-39a8-40f4-9630-478e51ada5b9\") " pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:55.964077 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.964051 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lbht\" (UniqueName: \"kubernetes.io/projected/bcdb6efe-39a8-40f4-9630-478e51ada5b9-kube-api-access-9lbht\") pod \"maas-api-56b457c467-xptbm\" (UID: \"bcdb6efe-39a8-40f4-9630-478e51ada5b9\") " pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:55.988798 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:55.988767 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:56.135748 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.135716 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-56b457c467-xptbm"] Jul 22 16:17:56.137055 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:17:56.137033 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcdb6efe_39a8_40f4_9630_478e51ada5b9.slice/crio-d3abb7448cdb45215e3375075b85220d5a0824201fd710f30a7afe78623653fa WatchSource:0}: Error finding container d3abb7448cdb45215e3375075b85220d5a0824201fd710f30a7afe78623653fa: Status 404 returned error can't find the container with id d3abb7448cdb45215e3375075b85220d5a0824201fd710f30a7afe78623653fa Jul 22 16:17:56.138495 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.138475 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:17:56.285414 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.285378 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" event={"ID":"bcdb6efe-39a8-40f4-9630-478e51ada5b9","Type":"ContainerStarted","Data":"d3abb7448cdb45215e3375075b85220d5a0824201fd710f30a7afe78623653fa"} Jul 22 16:17:56.292423 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.292389 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-8ddlf"] Jul 22 16:17:56.295781 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.295763 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" Jul 22 16:17:56.298965 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.298946 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 22 16:17:56.299337 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.299313 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9wflh\"" Jul 22 16:17:56.314129 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.314105 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-8ddlf"] Jul 22 16:17:56.342124 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.342095 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-x9p2f"] Jul 22 16:17:56.344795 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.344774 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" Jul 22 16:17:56.356521 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.356493 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-x9p2f"] Jul 22 16:17:56.457805 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.457763 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a2b024-28a6-419a-ba25-9b19d12f6174-plugins-config-volume\") pod \"payload-processing-5bf47cd988-8ddlf\" (UID: \"38a2b024-28a6-419a-ba25-9b19d12f6174\") " pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" Jul 22 16:17:56.458031 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.457944 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-895rl\" (UniqueName: \"kubernetes.io/projected/38a2b024-28a6-419a-ba25-9b19d12f6174-kube-api-access-895rl\") pod \"payload-processing-5bf47cd988-8ddlf\" (UID: \"38a2b024-28a6-419a-ba25-9b19d12f6174\") " pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" Jul 22 16:17:56.458031 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.457989 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1b4e347-134e-485c-aabc-8d0aa3d358d3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-x9p2f\" (UID: \"f1b4e347-134e-485c-aabc-8d0aa3d358d3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" Jul 22 16:17:56.458139 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.458027 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsxm9\" (UniqueName: \"kubernetes.io/projected/f1b4e347-134e-485c-aabc-8d0aa3d358d3-kube-api-access-rsxm9\") pod \"payload-pre-processing-7677b774bb-x9p2f\" (UID: \"f1b4e347-134e-485c-aabc-8d0aa3d358d3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" Jul 22 16:17:56.559245 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.559118 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-895rl\" (UniqueName: \"kubernetes.io/projected/38a2b024-28a6-419a-ba25-9b19d12f6174-kube-api-access-895rl\") pod \"payload-processing-5bf47cd988-8ddlf\" (UID: \"38a2b024-28a6-419a-ba25-9b19d12f6174\") " pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" Jul 22 16:17:56.559245 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.559184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1b4e347-134e-485c-aabc-8d0aa3d358d3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-x9p2f\" (UID: \"f1b4e347-134e-485c-aabc-8d0aa3d358d3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" Jul 22 16:17:56.559245 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.559209 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsxm9\" (UniqueName: \"kubernetes.io/projected/f1b4e347-134e-485c-aabc-8d0aa3d358d3-kube-api-access-rsxm9\") pod \"payload-pre-processing-7677b774bb-x9p2f\" (UID: \"f1b4e347-134e-485c-aabc-8d0aa3d358d3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" Jul 22 16:17:56.559552 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.559261 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a2b024-28a6-419a-ba25-9b19d12f6174-plugins-config-volume\") pod \"payload-processing-5bf47cd988-8ddlf\" (UID: \"38a2b024-28a6-419a-ba25-9b19d12f6174\") " pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" Jul 22 16:17:56.560006 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.559987 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a2b024-28a6-419a-ba25-9b19d12f6174-plugins-config-volume\") pod \"payload-processing-5bf47cd988-8ddlf\" (UID: \"38a2b024-28a6-419a-ba25-9b19d12f6174\") " pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" Jul 22 16:17:56.560006 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.559999 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1b4e347-134e-485c-aabc-8d0aa3d358d3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-x9p2f\" (UID: \"f1b4e347-134e-485c-aabc-8d0aa3d358d3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" Jul 22 16:17:56.568394 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.568363 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-895rl\" (UniqueName: \"kubernetes.io/projected/38a2b024-28a6-419a-ba25-9b19d12f6174-kube-api-access-895rl\") pod \"payload-processing-5bf47cd988-8ddlf\" (UID: \"38a2b024-28a6-419a-ba25-9b19d12f6174\") " pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" Jul 22 16:17:56.568518 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.568365 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsxm9\" (UniqueName: \"kubernetes.io/projected/f1b4e347-134e-485c-aabc-8d0aa3d358d3-kube-api-access-rsxm9\") pod \"payload-pre-processing-7677b774bb-x9p2f\" (UID: \"f1b4e347-134e-485c-aabc-8d0aa3d358d3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" Jul 22 16:17:56.607395 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.607363 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" Jul 22 16:17:56.655568 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.655373 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" Jul 22 16:17:56.810479 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.810417 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-x9p2f"] Jul 22 16:17:56.811941 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:17:56.811911 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b4e347_134e_485c_aabc_8d0aa3d358d3.slice/crio-e96ca30a2ddcc4c3b04a12c12e8ac2278dbfdac349412efcff4119e25a9088be WatchSource:0}: Error finding container e96ca30a2ddcc4c3b04a12c12e8ac2278dbfdac349412efcff4119e25a9088be: Status 404 returned error can't find the container with id e96ca30a2ddcc4c3b04a12c12e8ac2278dbfdac349412efcff4119e25a9088be Jul 22 16:17:56.994562 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:56.994508 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-8ddlf"] Jul 22 16:17:57.009745 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:17:57.009658 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a2b024_28a6_419a_ba25_9b19d12f6174.slice/crio-85472bdc193ed98bf83f6816849362cb8a865dd05663e27a7fd1d004db9a8982 WatchSource:0}: Error finding container 85472bdc193ed98bf83f6816849362cb8a865dd05663e27a7fd1d004db9a8982: Status 404 returned error can't find the container with id 85472bdc193ed98bf83f6816849362cb8a865dd05663e27a7fd1d004db9a8982 Jul 22 16:17:57.304847 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:57.304803 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" event={"ID":"f1b4e347-134e-485c-aabc-8d0aa3d358d3","Type":"ContainerStarted","Data":"e96ca30a2ddcc4c3b04a12c12e8ac2278dbfdac349412efcff4119e25a9088be"} Jul 22 16:17:57.306389 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:57.306338 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" event={"ID":"38a2b024-28a6-419a-ba25-9b19d12f6174","Type":"ContainerStarted","Data":"85472bdc193ed98bf83f6816849362cb8a865dd05663e27a7fd1d004db9a8982"} Jul 22 16:17:59.321793 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:59.321755 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" event={"ID":"bcdb6efe-39a8-40f4-9630-478e51ada5b9","Type":"ContainerStarted","Data":"ddd2a10d9688e1ec318fd5c628706dfba9a2243746d2edd041a1b8bdc42cc178"} Jul 22 16:17:59.322249 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:59.321993 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:17:59.323437 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:59.323408 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" event={"ID":"f1b4e347-134e-485c-aabc-8d0aa3d358d3","Type":"ContainerStarted","Data":"6593d642964b0dfdf8f4355b2adaaaa5378acae196096a6c1c29e61e195cc96e"} Jul 22 16:17:59.323566 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:59.323466 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" Jul 22 16:17:59.324947 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:59.324924 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" event={"ID":"38a2b024-28a6-419a-ba25-9b19d12f6174","Type":"ContainerStarted","Data":"7f06700736e3fbc21a57e759eab5e25f2551096903c6285220b8044d38e02ceb"} Jul 22 16:17:59.325075 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:59.325048 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" Jul 22 16:17:59.340142 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:59.340079 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" podStartSLOduration=2.10958586 podStartE2EDuration="4.340060502s" totalImagesPullingTime="2.230474642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:17:59.339911535 +0000 UTC m=+642.735453370" watchObservedRunningTime="2026-07-22 16:17:59.340060502 +0000 UTC m=+642.735602336" Jul 22 16:17:59.358963 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:59.358912 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" podStartSLOduration=1.007495529 podStartE2EDuration="3.358895834s" totalImagesPullingTime="2.351400305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:17:59.356464925 +0000 UTC m=+642.752006752" watchObservedRunningTime="2026-07-22 16:17:59.358895834 +0000 UTC m=+642.754437724" Jul 22 16:17:59.376775 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:17:59.376713 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" podStartSLOduration=1.18996253 podStartE2EDuration="3.376691177s" totalImagesPullingTime="2.186728647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:17:59.372587823 +0000 UTC m=+642.768129658" watchObservedRunningTime="2026-07-22 16:17:59.376691177 +0000 UTC m=+642.772233012" Jul 22 16:18:04.360341 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:04.360304 2564 generic.go:356] "Generic (PLEG): container finished" podID="3127e95b-74b6-4ef2-ace4-63bba9953ffc" containerID="eb26587767537242f30d60e3982bdff8f5b61bf292657a880f2243c36357a921" exitCode=0 Jul 22 16:18:04.360828 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:04.360384 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" event={"ID":"3127e95b-74b6-4ef2-ace4-63bba9953ffc","Type":"ContainerDied","Data":"eb26587767537242f30d60e3982bdff8f5b61bf292657a880f2243c36357a921"} Jul 22 16:18:04.360883 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:04.360866 2564 scope.go:117] "RemoveContainer" containerID="eb26587767537242f30d60e3982bdff8f5b61bf292657a880f2243c36357a921" Jul 22 16:18:04.362002 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:04.361964 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 22 16:18:04.362069 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:04.362053 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 22 16:18:04.362107 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:04.362097 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 22 16:18:05.336579 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:05.336544 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-56b457c467-xptbm" Jul 22 16:18:05.370577 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:05.370536 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" event={"ID":"3127e95b-74b6-4ef2-ace4-63bba9953ffc","Type":"ContainerStarted","Data":"83218f45fa60eb7daf03022a59f74f630050f6ce7cbb2eedffe1679f55003b99"} Jul 22 16:18:05.679678 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:05.679534 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:18:05.679870 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:05.679693 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fwbxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 16:18:05.679870 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:05.679747 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" podUID="3127e95b-74b6-4ef2-ace4-63bba9953ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 16:18:06.679929 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:06.679879 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fwbxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 16:18:06.680464 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:06.679960 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" podUID="3127e95b-74b6-4ef2-ace4-63bba9953ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 16:18:07.680230 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:07.680181 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fwbxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 16:18:07.680773 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:07.680256 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" podUID="3127e95b-74b6-4ef2-ace4-63bba9953ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 16:18:08.679454 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:08.679403 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fwbxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 16:18:08.679701 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:08.679475 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" podUID="3127e95b-74b6-4ef2-ace4-63bba9953ffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 16:18:09.683990 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:09.683959 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:18:09.684394 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:09.684226 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:18:09.685037 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:09.685019 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fwbxf" Jul 22 16:18:10.332846 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:10.332810 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-8ddlf" Jul 22 16:18:10.333027 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:10.332886 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-x9p2f" Jul 22 16:18:22.653224 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.653190 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq"] Jul 22 16:18:22.657057 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.657037 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.661097 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.661076 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 16:18:22.661501 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.661489 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 16:18:22.662440 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.662420 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-t84pc\"" Jul 22 16:18:22.662549 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.662504 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 22 16:18:22.674118 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.674089 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq"] Jul 22 16:18:22.730707 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.730669 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.730707 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.730706 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.730924 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.730734 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.730924 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.730763 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.730924 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.730862 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.731024 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.730955 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5ldh\" (UniqueName: \"kubernetes.io/projected/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-kube-api-access-d5ldh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.831687 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.831618 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5ldh\" (UniqueName: \"kubernetes.io/projected/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-kube-api-access-d5ldh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.831886 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.831713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.831886 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.831736 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.831886 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.831765 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.831886 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.831790 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.831886 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.831863 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.832259 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.832236 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.832370 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.832279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.832370 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.832314 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.834179 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.834157 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.834472 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.834453 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.841750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.841728 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5ldh\" (UniqueName: \"kubernetes.io/projected/ab10cd92-a9f3-4290-99f7-6c5b5fbfe419-kube-api-access-d5ldh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq\" (UID: \"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:22.968502 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:22.968411 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:23.116848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:23.116823 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq"] Jul 22 16:18:23.118728 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:18:23.118698 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab10cd92_a9f3_4290_99f7_6c5b5fbfe419.slice/crio-eeb42682065a26c8d4e80ab76d007fad4d0d25338038019514db2b82776efcbe WatchSource:0}: Error finding container eeb42682065a26c8d4e80ab76d007fad4d0d25338038019514db2b82776efcbe: Status 404 returned error can't find the container with id eeb42682065a26c8d4e80ab76d007fad4d0d25338038019514db2b82776efcbe Jul 22 16:18:23.495070 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:23.495027 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" event={"ID":"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419","Type":"ContainerStarted","Data":"eeb42682065a26c8d4e80ab76d007fad4d0d25338038019514db2b82776efcbe"} Jul 22 16:18:24.502994 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:24.502948 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" event={"ID":"ab10cd92-a9f3-4290-99f7-6c5b5fbfe419","Type":"ContainerStarted","Data":"f2a1a76b0de43637fd90e89815025dd96cb3a35674e8266929d4a82515816674"} Jul 22 16:18:24.523229 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:24.523174 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" podStartSLOduration=1.3665547679999999 podStartE2EDuration="2.523157873s" totalImagesPullingTime="1.156603105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:18:24.522109857 +0000 UTC m=+667.917651691" watchObservedRunningTime="2026-07-22 16:18:24.523157873 +0000 UTC m=+667.918699743" Jul 22 16:18:30.733378 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.733330 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd"] Jul 22 16:18:30.736252 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.736227 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.739086 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.739063 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 22 16:18:30.747746 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.747717 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd"] Jul 22 16:18:30.811816 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.811779 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.811816 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.811815 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.812033 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.811924 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.812033 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.811941 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.812033 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.811971 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwrhx\" (UniqueName: \"kubernetes.io/projected/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-kube-api-access-qwrhx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.812033 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.811999 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.913250 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.913211 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwrhx\" (UniqueName: \"kubernetes.io/projected/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-kube-api-access-qwrhx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.913428 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.913261 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.913428 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.913303 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.913428 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.913320 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.913428 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.913383 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.913428 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.913406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.913809 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.913784 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.913859 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.913795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.913859 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.913848 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.915747 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.915722 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.915999 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.915983 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:30.922769 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:30.922743 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwrhx\" (UniqueName: \"kubernetes.io/projected/dca69f5a-b04e-433f-be7a-dc9d17e6e5b9-kube-api-access-qwrhx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jncjd\" (UID: \"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:31.048848 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:31.048752 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:31.194974 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:31.194938 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd"] Jul 22 16:18:31.197235 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:18:31.197201 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca69f5a_b04e_433f_be7a_dc9d17e6e5b9.slice/crio-dacbdfdf2428cdf6badebce37f52629fdaebddc84e41ae54b98af8831143fb29 WatchSource:0}: Error finding container dacbdfdf2428cdf6badebce37f52629fdaebddc84e41ae54b98af8831143fb29: Status 404 returned error can't find the container with id dacbdfdf2428cdf6badebce37f52629fdaebddc84e41ae54b98af8831143fb29 Jul 22 16:18:31.548888 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:31.548854 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" event={"ID":"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9","Type":"ContainerStarted","Data":"dacbdfdf2428cdf6badebce37f52629fdaebddc84e41ae54b98af8831143fb29"} Jul 22 16:18:32.555858 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:32.555819 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" event={"ID":"dca69f5a-b04e-433f-be7a-dc9d17e6e5b9","Type":"ContainerStarted","Data":"94fb91f7c7da0ecedb54d47e57d9328c61de1c4e97c5eab24366a4831b12b3b1"} Jul 22 16:18:32.579707 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:32.579654 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" podStartSLOduration=2.242681458 podStartE2EDuration="2.579636885s" totalImagesPullingTime="336.955427ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:18:32.578313982 +0000 UTC m=+675.973855816" watchObservedRunningTime="2026-07-22 16:18:32.579636885 +0000 UTC m=+675.975178711" Jul 22 16:18:32.969558 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:32.969462 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:32.969558 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:32.969549 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:32.982912 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:32.982881 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:33.572868 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:33.572837 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq" Jul 22 16:18:41.049183 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:41.049136 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:41.049746 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:41.049215 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:41.065601 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:41.065567 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:18:41.617322 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:18:41.617288 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jncjd" Jul 22 16:19:16.913388 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:16.913300 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx"] Jul 22 16:19:16.916384 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:16.916361 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:16.920547 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:16.920520 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 22 16:19:16.951755 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:16.951707 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx"] Jul 22 16:19:17.058200 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.058164 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.058371 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.058221 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg25k\" (UniqueName: \"kubernetes.io/projected/71488d8e-3c4d-43ae-b926-f44744f960b3-kube-api-access-kg25k\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.058371 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.058280 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.058371 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.058296 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71488d8e-3c4d-43ae-b926-f44744f960b3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.058472 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.058379 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.058472 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.058422 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.159460 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.159426 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.159678 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.159469 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.159678 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.159503 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.159678 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.159538 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg25k\" (UniqueName: \"kubernetes.io/projected/71488d8e-3c4d-43ae-b926-f44744f960b3-kube-api-access-kg25k\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.159678 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.159562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.159678 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.159579 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71488d8e-3c4d-43ae-b926-f44744f960b3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.159957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.159886 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.159957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.159929 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.160069 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.160021 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.161793 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.161768 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71488d8e-3c4d-43ae-b926-f44744f960b3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.162452 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.162432 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 22 16:19:17.170462 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.170397 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg25k\" (UniqueName: \"kubernetes.io/projected/71488d8e-3c4d-43ae-b926-f44744f960b3-kube-api-access-kg25k\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.172085 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.172064 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71488d8e-3c4d-43ae-b926-f44744f960b3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx\" (UID: \"71488d8e-3c4d-43ae-b926-f44744f960b3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.227140 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.227103 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:17.369955 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.369931 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx"] Jul 22 16:19:17.371793 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:19:17.371766 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71488d8e_3c4d_43ae_b926_f44744f960b3.slice/crio-2717029e3d4045c2c4dd656b46a9321a74d32347c0dabf998c72fa1ccd8ce80f WatchSource:0}: Error finding container 2717029e3d4045c2c4dd656b46a9321a74d32347c0dabf998c72fa1ccd8ce80f: Status 404 returned error can't find the container with id 2717029e3d4045c2c4dd656b46a9321a74d32347c0dabf998c72fa1ccd8ce80f Jul 22 16:19:17.822080 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.821976 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" event={"ID":"71488d8e-3c4d-43ae-b926-f44744f960b3","Type":"ContainerStarted","Data":"a955c3816381934006dfe4d9821c959e55e4cf84cc24f1414a8f393aa5b134a9"} Jul 22 16:19:17.822080 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.822022 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" event={"ID":"71488d8e-3c4d-43ae-b926-f44744f960b3","Type":"ContainerStarted","Data":"2717029e3d4045c2c4dd656b46a9321a74d32347c0dabf998c72fa1ccd8ce80f"} Jul 22 16:19:17.841821 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:17.841763 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" podStartSLOduration=1.653364701 podStartE2EDuration="1.841746898s" totalImagesPullingTime="188.382197ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:19:17.839777675 +0000 UTC m=+721.235319511" watchObservedRunningTime="2026-07-22 16:19:17.841746898 +0000 UTC m=+721.237288733" Jul 22 16:19:27.227276 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:27.227231 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:27.227719 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:27.227288 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:27.240178 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:27.240153 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:19:27.892508 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:19:27.892479 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx" Jul 22 16:20:16.018053 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.018019 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6455bbf649-9xppr"] Jul 22 16:20:16.021498 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.021476 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6455bbf649-9xppr" Jul 22 16:20:16.025592 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.025562 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 22 16:20:16.025761 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.025613 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gn4s2\"" Jul 22 16:20:16.028953 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.028927 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6455bbf649-9xppr"] Jul 22 16:20:16.115408 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.115363 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07fa8ebe-6ac0-4a0c-aa04-11e9fa708450-tls-cert\") pod \"authorino-6455bbf649-9xppr\" (UID: \"07fa8ebe-6ac0-4a0c-aa04-11e9fa708450\") " pod="kuadrant-system/authorino-6455bbf649-9xppr" Jul 22 16:20:16.115610 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.115457 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b44t6\" (UniqueName: \"kubernetes.io/projected/07fa8ebe-6ac0-4a0c-aa04-11e9fa708450-kube-api-access-b44t6\") pod \"authorino-6455bbf649-9xppr\" (UID: \"07fa8ebe-6ac0-4a0c-aa04-11e9fa708450\") " pod="kuadrant-system/authorino-6455bbf649-9xppr" Jul 22 16:20:16.216889 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.216851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07fa8ebe-6ac0-4a0c-aa04-11e9fa708450-tls-cert\") pod \"authorino-6455bbf649-9xppr\" (UID: \"07fa8ebe-6ac0-4a0c-aa04-11e9fa708450\") " pod="kuadrant-system/authorino-6455bbf649-9xppr" Jul 22 16:20:16.217121 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.216964 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b44t6\" (UniqueName: \"kubernetes.io/projected/07fa8ebe-6ac0-4a0c-aa04-11e9fa708450-kube-api-access-b44t6\") pod \"authorino-6455bbf649-9xppr\" (UID: \"07fa8ebe-6ac0-4a0c-aa04-11e9fa708450\") " pod="kuadrant-system/authorino-6455bbf649-9xppr" Jul 22 16:20:16.219310 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.219279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07fa8ebe-6ac0-4a0c-aa04-11e9fa708450-tls-cert\") pod \"authorino-6455bbf649-9xppr\" (UID: \"07fa8ebe-6ac0-4a0c-aa04-11e9fa708450\") " pod="kuadrant-system/authorino-6455bbf649-9xppr" Jul 22 16:20:16.230055 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.230022 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b44t6\" (UniqueName: \"kubernetes.io/projected/07fa8ebe-6ac0-4a0c-aa04-11e9fa708450-kube-api-access-b44t6\") pod \"authorino-6455bbf649-9xppr\" (UID: \"07fa8ebe-6ac0-4a0c-aa04-11e9fa708450\") " pod="kuadrant-system/authorino-6455bbf649-9xppr" Jul 22 16:20:16.333109 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.333037 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6455bbf649-9xppr" Jul 22 16:20:16.492153 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:16.492124 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6455bbf649-9xppr"] Jul 22 16:20:16.494402 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:20:16.494366 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07fa8ebe_6ac0_4a0c_aa04_11e9fa708450.slice/crio-605adb7b899cfe4f20b443ecaf6536bc3ef3e88aaedc130a24026a27bca8c2ff WatchSource:0}: Error finding container 605adb7b899cfe4f20b443ecaf6536bc3ef3e88aaedc130a24026a27bca8c2ff: Status 404 returned error can't find the container with id 605adb7b899cfe4f20b443ecaf6536bc3ef3e88aaedc130a24026a27bca8c2ff Jul 22 16:20:17.179207 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:17.179164 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6455bbf649-9xppr" event={"ID":"07fa8ebe-6ac0-4a0c-aa04-11e9fa708450","Type":"ContainerStarted","Data":"76976b5f2865300070afa53b29e7b86de750b3186ed50ef574e42878d37591fe"} Jul 22 16:20:17.179207 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:17.179211 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6455bbf649-9xppr" event={"ID":"07fa8ebe-6ac0-4a0c-aa04-11e9fa708450","Type":"ContainerStarted","Data":"605adb7b899cfe4f20b443ecaf6536bc3ef3e88aaedc130a24026a27bca8c2ff"} Jul 22 16:20:17.204656 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:20:17.204563 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6455bbf649-9xppr" podStartSLOduration=1.631529367 podStartE2EDuration="2.204544023s" totalImagesPullingTime="573.014656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:20:17.20065987 +0000 UTC m=+780.596201704" watchObservedRunningTime="2026-07-22 16:20:17.204544023 +0000 UTC m=+780.600085910" Jul 22 16:21:04.176261 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.176219 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6848f74968-d46xl"] Jul 22 16:21:04.176862 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.176548 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6848f74968-d46xl" podUID="e581dbea-b9c5-4cca-875d-e2759f57cc46" containerName="manager" containerID="cri-o://daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96" gracePeriod=30 Jul 22 16:21:04.441183 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.441112 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:21:04.471618 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.471577 2564 generic.go:356] "Generic (PLEG): container finished" podID="e581dbea-b9c5-4cca-875d-e2759f57cc46" containerID="daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96" exitCode=0 Jul 22 16:21:04.471821 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.471649 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6848f74968-d46xl" Jul 22 16:21:04.471821 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.471663 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6848f74968-d46xl" event={"ID":"e581dbea-b9c5-4cca-875d-e2759f57cc46","Type":"ContainerDied","Data":"daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96"} Jul 22 16:21:04.471821 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.471730 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6848f74968-d46xl" event={"ID":"e581dbea-b9c5-4cca-875d-e2759f57cc46","Type":"ContainerDied","Data":"95f8a6f3a537e57888701edbe1368187bacc6a84ecf02e034e000d2a8f57a4b4"} Jul 22 16:21:04.471821 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.471757 2564 scope.go:117] "RemoveContainer" containerID="daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96" Jul 22 16:21:04.482013 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.481990 2564 scope.go:117] "RemoveContainer" containerID="daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96" Jul 22 16:21:04.482334 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:21:04.482312 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96\": container with ID starting with daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96 not found: ID does not exist" containerID="daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96" Jul 22 16:21:04.482442 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.482342 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96"} err="failed to get container status \"daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96\": rpc error: code = NotFound desc = could not find container \"daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96\": container with ID starting with daa48142525aea9d709646d90a9d68c1ecbe6b0655c79b6bb099898ee8f4da96 not found: ID does not exist" Jul 22 16:21:04.491273 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.491227 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2b2c\" (UniqueName: \"kubernetes.io/projected/e581dbea-b9c5-4cca-875d-e2759f57cc46-kube-api-access-n2b2c\") pod \"e581dbea-b9c5-4cca-875d-e2759f57cc46\" (UID: \"e581dbea-b9c5-4cca-875d-e2759f57cc46\") " Jul 22 16:21:04.491409 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.491323 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e581dbea-b9c5-4cca-875d-e2759f57cc46-webhook-cert\") pod \"e581dbea-b9c5-4cca-875d-e2759f57cc46\" (UID: \"e581dbea-b9c5-4cca-875d-e2759f57cc46\") " Jul 22 16:21:04.493744 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.493712 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e581dbea-b9c5-4cca-875d-e2759f57cc46-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e581dbea-b9c5-4cca-875d-e2759f57cc46" (UID: "e581dbea-b9c5-4cca-875d-e2759f57cc46"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:21:04.493863 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.493849 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e581dbea-b9c5-4cca-875d-e2759f57cc46-kube-api-access-n2b2c" (OuterVolumeSpecName: "kube-api-access-n2b2c") pod "e581dbea-b9c5-4cca-875d-e2759f57cc46" (UID: "e581dbea-b9c5-4cca-875d-e2759f57cc46"). InnerVolumeSpecName "kube-api-access-n2b2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:21:04.592610 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.592572 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2b2c\" (UniqueName: \"kubernetes.io/projected/e581dbea-b9c5-4cca-875d-e2759f57cc46-kube-api-access-n2b2c\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:21:04.592610 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.592603 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e581dbea-b9c5-4cca-875d-e2759f57cc46-webhook-cert\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:21:04.797002 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.796971 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6848f74968-d46xl"] Jul 22 16:21:04.802170 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:04.802141 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6848f74968-d46xl"] Jul 22 16:21:05.129480 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:21:05.129400 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e581dbea-b9c5-4cca-875d-e2759f57cc46" path="/var/lib/kubelet/pods/e581dbea-b9c5-4cca-875d-e2759f57cc46/volumes" Jul 22 16:27:24.045018 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:24.044980 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h"] Jul 22 16:27:24.045587 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:24.045248 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" podUID="eb6723e3-baac-40a3-bfef-aa44a9f62d64" containerName="manager" containerID="cri-o://3a4bf4be117f54232706d06d5ee62c96089567606bdd48c860a373334aed759e" gracePeriod=10 Jul 22 16:27:26.764250 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:26.764216 2564 generic.go:356] "Generic (PLEG): container finished" podID="eb6723e3-baac-40a3-bfef-aa44a9f62d64" containerID="3a4bf4be117f54232706d06d5ee62c96089567606bdd48c860a373334aed759e" exitCode=0 Jul 22 16:27:26.764753 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:26.764289 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" event={"ID":"eb6723e3-baac-40a3-bfef-aa44a9f62d64","Type":"ContainerDied","Data":"3a4bf4be117f54232706d06d5ee62c96089567606bdd48c860a373334aed759e"} Jul 22 16:27:26.816111 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:26.816086 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:27:26.859889 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:26.859791 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb6723e3-baac-40a3-bfef-aa44a9f62d64-extensions-socket-volume\") pod \"eb6723e3-baac-40a3-bfef-aa44a9f62d64\" (UID: \"eb6723e3-baac-40a3-bfef-aa44a9f62d64\") " Jul 22 16:27:26.860062 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:26.859898 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2czl\" (UniqueName: \"kubernetes.io/projected/eb6723e3-baac-40a3-bfef-aa44a9f62d64-kube-api-access-h2czl\") pod \"eb6723e3-baac-40a3-bfef-aa44a9f62d64\" (UID: \"eb6723e3-baac-40a3-bfef-aa44a9f62d64\") " Jul 22 16:27:26.860182 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:26.860159 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6723e3-baac-40a3-bfef-aa44a9f62d64-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "eb6723e3-baac-40a3-bfef-aa44a9f62d64" (UID: "eb6723e3-baac-40a3-bfef-aa44a9f62d64"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:27:26.862109 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:26.862077 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6723e3-baac-40a3-bfef-aa44a9f62d64-kube-api-access-h2czl" (OuterVolumeSpecName: "kube-api-access-h2czl") pod "eb6723e3-baac-40a3-bfef-aa44a9f62d64" (UID: "eb6723e3-baac-40a3-bfef-aa44a9f62d64"). InnerVolumeSpecName "kube-api-access-h2czl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:27:26.961733 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:26.961693 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2czl\" (UniqueName: \"kubernetes.io/projected/eb6723e3-baac-40a3-bfef-aa44a9f62d64-kube-api-access-h2czl\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:27:26.961733 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:26.961728 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb6723e3-baac-40a3-bfef-aa44a9f62d64-extensions-socket-volume\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:27:27.772884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:27.772849 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" Jul 22 16:27:27.772884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:27.772866 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h" event={"ID":"eb6723e3-baac-40a3-bfef-aa44a9f62d64","Type":"ContainerDied","Data":"fe54549698bf32f3a79a66d6931baa2139c1eae95ac4e7e7fdd2766355155e26"} Jul 22 16:27:27.773374 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:27.772961 2564 scope.go:117] "RemoveContainer" containerID="3a4bf4be117f54232706d06d5ee62c96089567606bdd48c860a373334aed759e" Jul 22 16:27:27.794491 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:27.794458 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h"] Jul 22 16:27:27.799425 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:27.799399 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bz69h"] Jul 22 16:27:29.129393 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:27:29.129357 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb6723e3-baac-40a3-bfef-aa44a9f62d64" path="/var/lib/kubelet/pods/eb6723e3-baac-40a3-bfef-aa44a9f62d64/volumes" Jul 22 16:28:25.199448 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.199410 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2"] Jul 22 16:28:25.199935 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.199860 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb6723e3-baac-40a3-bfef-aa44a9f62d64" containerName="manager" Jul 22 16:28:25.199935 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.199871 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb6723e3-baac-40a3-bfef-aa44a9f62d64" containerName="manager" Jul 22 16:28:25.199935 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.199888 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e581dbea-b9c5-4cca-875d-e2759f57cc46" containerName="manager" Jul 22 16:28:25.199935 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.199893 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e581dbea-b9c5-4cca-875d-e2759f57cc46" containerName="manager" Jul 22 16:28:25.200088 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.199956 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e581dbea-b9c5-4cca-875d-e2759f57cc46" containerName="manager" Jul 22 16:28:25.200088 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.199966 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb6723e3-baac-40a3-bfef-aa44a9f62d64" containerName="manager" Jul 22 16:28:25.204187 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.204170 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" Jul 22 16:28:25.207897 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.207878 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fpl8z\"" Jul 22 16:28:25.225664 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.225615 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2"] Jul 22 16:28:25.327329 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.327295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e55314-6e1f-4640-a6c4-3a44e4967b5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vkqh2\" (UID: \"f9e55314-6e1f-4640-a6c4-3a44e4967b5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" Jul 22 16:28:25.327514 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.327345 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwkv9\" (UniqueName: \"kubernetes.io/projected/f9e55314-6e1f-4640-a6c4-3a44e4967b5f-kube-api-access-cwkv9\") pod \"kuadrant-operator-controller-manager-86878c769f-vkqh2\" (UID: \"f9e55314-6e1f-4640-a6c4-3a44e4967b5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" Jul 22 16:28:25.428233 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.428166 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e55314-6e1f-4640-a6c4-3a44e4967b5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vkqh2\" (UID: \"f9e55314-6e1f-4640-a6c4-3a44e4967b5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" Jul 22 16:28:25.428233 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.428244 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwkv9\" (UniqueName: \"kubernetes.io/projected/f9e55314-6e1f-4640-a6c4-3a44e4967b5f-kube-api-access-cwkv9\") pod \"kuadrant-operator-controller-manager-86878c769f-vkqh2\" (UID: \"f9e55314-6e1f-4640-a6c4-3a44e4967b5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" Jul 22 16:28:25.428714 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.428691 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e55314-6e1f-4640-a6c4-3a44e4967b5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vkqh2\" (UID: \"f9e55314-6e1f-4640-a6c4-3a44e4967b5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" Jul 22 16:28:25.438801 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.438770 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwkv9\" (UniqueName: \"kubernetes.io/projected/f9e55314-6e1f-4640-a6c4-3a44e4967b5f-kube-api-access-cwkv9\") pod \"kuadrant-operator-controller-manager-86878c769f-vkqh2\" (UID: \"f9e55314-6e1f-4640-a6c4-3a44e4967b5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" Jul 22 16:28:25.514135 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.514105 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" Jul 22 16:28:25.674238 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:25.674207 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2"] Jul 22 16:28:25.675662 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:28:25.675618 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e55314_6e1f_4640_a6c4_3a44e4967b5f.slice/crio-3c1a22f5bce9b5fd3efe2c48e6b78fdd76e2359dad5e76763bef0efe5c2bf3c1 WatchSource:0}: Error finding container 3c1a22f5bce9b5fd3efe2c48e6b78fdd76e2359dad5e76763bef0efe5c2bf3c1: Status 404 returned error can't find the container with id 3c1a22f5bce9b5fd3efe2c48e6b78fdd76e2359dad5e76763bef0efe5c2bf3c1 Jul 22 16:28:26.102900 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:26.102856 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" event={"ID":"f9e55314-6e1f-4640-a6c4-3a44e4967b5f","Type":"ContainerStarted","Data":"4e237e3e4fb511e46be6173cdd6a9ada72ab9a5ef0b3aba23866a1ddc401df48"} Jul 22 16:28:26.103113 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:26.102915 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" Jul 22 16:28:26.103113 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:26.102938 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" event={"ID":"f9e55314-6e1f-4640-a6c4-3a44e4967b5f","Type":"ContainerStarted","Data":"3c1a22f5bce9b5fd3efe2c48e6b78fdd76e2359dad5e76763bef0efe5c2bf3c1"} Jul 22 16:28:26.127416 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:26.127366 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" podStartSLOduration=1.127349801 podStartE2EDuration="1.127349801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:28:26.126199057 +0000 UTC m=+1269.521740889" watchObservedRunningTime="2026-07-22 16:28:26.127349801 +0000 UTC m=+1269.522891634" Jul 22 16:28:37.111016 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:28:37.110984 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkqh2" Jul 22 16:42:11.149396 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.149354 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp"] Jul 22 16:42:11.153131 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.153110 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp" Jul 22 16:42:11.155705 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.155684 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-phfdj\"" Jul 22 16:42:11.162452 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.162419 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp"] Jul 22 16:42:11.217387 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.217346 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df52a478-02d3-4770-a32e-a18498eafe5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4972458a-xbwlp\" (UID: \"df52a478-02d3-4770-a32e-a18498eafe5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp" Jul 22 16:42:11.318277 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.318231 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df52a478-02d3-4770-a32e-a18498eafe5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4972458a-xbwlp\" (UID: \"df52a478-02d3-4770-a32e-a18498eafe5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp" Jul 22 16:42:11.318925 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.318905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df52a478-02d3-4770-a32e-a18498eafe5d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4972458a-xbwlp\" (UID: \"df52a478-02d3-4770-a32e-a18498eafe5d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp" Jul 22 16:42:11.471558 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.471524 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp" Jul 22 16:42:11.621203 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.621167 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp"] Jul 22 16:42:11.622536 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:42:11.622510 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf52a478_02d3_4770_a32e_a18498eafe5d.slice/crio-64068d06b0de39b0b0ee265ef5b8b3f8f84c2c43ec7cb1656755c0a1e9c76abf WatchSource:0}: Error finding container 64068d06b0de39b0b0ee265ef5b8b3f8f84c2c43ec7cb1656755c0a1e9c76abf: Status 404 returned error can't find the container with id 64068d06b0de39b0b0ee265ef5b8b3f8f84c2c43ec7cb1656755c0a1e9c76abf Jul 22 16:42:11.624400 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.624382 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:42:11.930906 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:11.930810 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp" event={"ID":"df52a478-02d3-4770-a32e-a18498eafe5d","Type":"ContainerStarted","Data":"64068d06b0de39b0b0ee265ef5b8b3f8f84c2c43ec7cb1656755c0a1e9c76abf"} Jul 22 16:42:14.950361 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:14.950270 2564 generic.go:356] "Generic (PLEG): container finished" podID="df52a478-02d3-4770-a32e-a18498eafe5d" containerID="647bc88c1363d32641d14ee51042e9cf1da0018dc3cd60a205dc58d9887b9629" exitCode=0 Jul 22 16:42:14.952796 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:14.950351 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp" event={"ID":"df52a478-02d3-4770-a32e-a18498eafe5d","Type":"ContainerDied","Data":"647bc88c1363d32641d14ee51042e9cf1da0018dc3cd60a205dc58d9887b9629"} Jul 22 16:42:16.104689 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:16.104660 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp" Jul 22 16:42:16.267476 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:16.267434 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df52a478-02d3-4770-a32e-a18498eafe5d-maas-api-service-ca\") pod \"df52a478-02d3-4770-a32e-a18498eafe5d\" (UID: \"df52a478-02d3-4770-a32e-a18498eafe5d\") " Jul 22 16:42:16.267843 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:16.267819 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df52a478-02d3-4770-a32e-a18498eafe5d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "df52a478-02d3-4770-a32e-a18498eafe5d" (UID: "df52a478-02d3-4770-a32e-a18498eafe5d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:42:16.368701 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:16.368658 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df52a478-02d3-4770-a32e-a18498eafe5d-maas-api-service-ca\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:42:16.969245 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:16.969212 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp" Jul 22 16:42:16.969423 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:16.969255 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp" event={"ID":"df52a478-02d3-4770-a32e-a18498eafe5d","Type":"ContainerDied","Data":"64068d06b0de39b0b0ee265ef5b8b3f8f84c2c43ec7cb1656755c0a1e9c76abf"} Jul 22 16:42:16.969423 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:16.969325 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64068d06b0de39b0b0ee265ef5b8b3f8f84c2c43ec7cb1656755c0a1e9c76abf" Jul 22 16:42:31.236698 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:31.236658 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp"] Jul 22 16:42:31.239000 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:31.238975 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4972458a-xbwlp"] Jul 22 16:42:33.129986 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:42:33.129949 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df52a478-02d3-4770-a32e-a18498eafe5d" path="/var/lib/kubelet/pods/df52a478-02d3-4770-a32e-a18498eafe5d/volumes" Jul 22 16:45:00.138304 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.138267 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7"] Jul 22 16:45:00.138783 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.138722 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df52a478-02d3-4770-a32e-a18498eafe5d" containerName="revoke-keys" Jul 22 16:45:00.138783 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.138734 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="df52a478-02d3-4770-a32e-a18498eafe5d" containerName="revoke-keys" Jul 22 16:45:00.138860 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.138820 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="df52a478-02d3-4770-a32e-a18498eafe5d" containerName="revoke-keys" Jul 22 16:45:00.141804 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.141783 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7" Jul 22 16:45:00.148045 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.148022 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-phfdj\"" Jul 22 16:45:00.151867 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.151840 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7"] Jul 22 16:45:00.152508 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.152489 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7" Jul 22 16:45:00.307947 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.307919 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7"] Jul 22 16:45:00.309293 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:45:00.309262 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf72b16b_3e18_4ad6_94dd_fcfd946a68cf.slice/crio-21f71f3974ee01bf5f7e35c973cfbb327b689991c13dbe6bfaefade3eb15f8f6 WatchSource:0}: Error finding container 21f71f3974ee01bf5f7e35c973cfbb327b689991c13dbe6bfaefade3eb15f8f6: Status 404 returned error can't find the container with id 21f71f3974ee01bf5f7e35c973cfbb327b689991c13dbe6bfaefade3eb15f8f6 Jul 22 16:45:00.954306 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.954206 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7" event={"ID":"cf72b16b-3e18-4ad6-94dd-fcfd946a68cf","Type":"ContainerStarted","Data":"1bbf9f131027c6e7de65c89ea9cdb21c02032efa64fef5b158d4b7e1b00970f3"} Jul 22 16:45:00.954456 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.954310 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7" event={"ID":"cf72b16b-3e18-4ad6-94dd-fcfd946a68cf","Type":"ContainerStarted","Data":"21f71f3974ee01bf5f7e35c973cfbb327b689991c13dbe6bfaefade3eb15f8f6"} Jul 22 16:45:00.974375 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:00.974314 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7" podStartSLOduration=0.974296443 podStartE2EDuration="974.296443ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:45:00.973450929 +0000 UTC m=+2264.368992762" watchObservedRunningTime="2026-07-22 16:45:00.974296443 +0000 UTC m=+2264.369838275" Jul 22 16:45:01.962721 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:01.962681 2564 generic.go:356] "Generic (PLEG): container finished" podID="cf72b16b-3e18-4ad6-94dd-fcfd946a68cf" containerID="1bbf9f131027c6e7de65c89ea9cdb21c02032efa64fef5b158d4b7e1b00970f3" exitCode=0 Jul 22 16:45:01.963174 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:01.962733 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7" event={"ID":"cf72b16b-3e18-4ad6-94dd-fcfd946a68cf","Type":"ContainerDied","Data":"1bbf9f131027c6e7de65c89ea9cdb21c02032efa64fef5b158d4b7e1b00970f3"} Jul 22 16:45:03.104795 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:03.104769 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7" Jul 22 16:45:03.976633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:03.976578 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7" event={"ID":"cf72b16b-3e18-4ad6-94dd-fcfd946a68cf","Type":"ContainerDied","Data":"21f71f3974ee01bf5f7e35c973cfbb327b689991c13dbe6bfaefade3eb15f8f6"} Jul 22 16:45:03.976633 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:03.976603 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7" Jul 22 16:45:03.976859 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:45:03.976700 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21f71f3974ee01bf5f7e35c973cfbb327b689991c13dbe6bfaefade3eb15f8f6" Jul 22 16:46:15.508125 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:46:15.508028 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7"] Jul 22 16:46:15.519929 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:46:15.519891 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-fa170d-29745645-shlh7"] Jul 22 16:46:17.131267 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:46:17.131234 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf72b16b-3e18-4ad6-94dd-fcfd946a68cf" path="/var/lib/kubelet/pods/cf72b16b-3e18-4ad6-94dd-fcfd946a68cf/volumes" Jul 22 16:48:17.360366 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:48:17.360335 2564 scope.go:117] "RemoveContainer" containerID="647bc88c1363d32641d14ee51042e9cf1da0018dc3cd60a205dc58d9887b9629" Jul 22 16:51:17.404323 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:51:17.404292 2564 scope.go:117] "RemoveContainer" containerID="1bbf9f131027c6e7de65c89ea9cdb21c02032efa64fef5b158d4b7e1b00970f3" Jul 22 16:54:19.347535 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.347486 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7694-140398118560432"] Jul 22 16:54:19.348205 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.348147 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf72b16b-3e18-4ad6-94dd-fcfd946a68cf" containerName="cleanup" Jul 22 16:54:19.348205 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.348172 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf72b16b-3e18-4ad6-94dd-fcfd946a68cf" containerName="cleanup" Jul 22 16:54:19.348328 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.348303 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf72b16b-3e18-4ad6-94dd-fcfd946a68cf" containerName="cleanup" Jul 22 16:54:19.351524 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.351498 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118560432" Jul 22 16:54:19.354184 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.354146 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tjvgd\"" Jul 22 16:54:19.360960 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.360933 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-140398118560432"] Jul 22 16:54:19.523993 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.523950 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65ld6\" (UniqueName: \"kubernetes.io/projected/091d2544-e854-405a-abb0-a5021aacb771-kube-api-access-65ld6\") pod \"test-curl-7694-140398118560432\" (UID: \"091d2544-e854-405a-abb0-a5021aacb771\") " pod="opendatahub/test-curl-7694-140398118560432" Jul 22 16:54:19.625155 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.625050 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65ld6\" (UniqueName: \"kubernetes.io/projected/091d2544-e854-405a-abb0-a5021aacb771-kube-api-access-65ld6\") pod \"test-curl-7694-140398118560432\" (UID: \"091d2544-e854-405a-abb0-a5021aacb771\") " pod="opendatahub/test-curl-7694-140398118560432" Jul 22 16:54:19.634721 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.634681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65ld6\" (UniqueName: \"kubernetes.io/projected/091d2544-e854-405a-abb0-a5021aacb771-kube-api-access-65ld6\") pod \"test-curl-7694-140398118560432\" (UID: \"091d2544-e854-405a-abb0-a5021aacb771\") " pod="opendatahub/test-curl-7694-140398118560432" Jul 22 16:54:19.662999 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.662949 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118560432" Jul 22 16:54:19.814814 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.814778 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-140398118560432"] Jul 22 16:54:19.816778 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:54:19.816740 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod091d2544_e854_405a_abb0_a5021aacb771.slice/crio-ef71d7c6df2e805fef367d7cb75c7998fd4ca3068cdf424a0513e3302535b3f4 WatchSource:0}: Error finding container ef71d7c6df2e805fef367d7cb75c7998fd4ca3068cdf424a0513e3302535b3f4: Status 404 returned error can't find the container with id ef71d7c6df2e805fef367d7cb75c7998fd4ca3068cdf424a0513e3302535b3f4 Jul 22 16:54:19.818119 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:19.818095 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:54:20.215775 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:20.215724 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-140398118560432" event={"ID":"091d2544-e854-405a-abb0-a5021aacb771","Type":"ContainerStarted","Data":"ef71d7c6df2e805fef367d7cb75c7998fd4ca3068cdf424a0513e3302535b3f4"} Jul 22 16:54:21.223010 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:21.222958 2564 generic.go:356] "Generic (PLEG): container finished" podID="091d2544-e854-405a-abb0-a5021aacb771" containerID="6ce5672c480e0abc173fcf62762a6396d9ff8243d9a6dd1c564558dbfeead7a1" exitCode=0 Jul 22 16:54:21.223486 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:21.223023 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-140398118560432" event={"ID":"091d2544-e854-405a-abb0-a5021aacb771","Type":"ContainerDied","Data":"6ce5672c480e0abc173fcf62762a6396d9ff8243d9a6dd1c564558dbfeead7a1"} Jul 22 16:54:22.381486 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.381454 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118560432" Jul 22 16:54:22.427370 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.427339 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7694-140398118560432_091d2544-e854-405a-abb0-a5021aacb771/test-curl-7694-140398118560432/0.log" Jul 22 16:54:22.459218 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.459171 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7694-140398118560432"] Jul 22 16:54:22.467146 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.467115 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7694-140398118560432"] Jul 22 16:54:22.553878 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.553786 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65ld6\" (UniqueName: \"kubernetes.io/projected/091d2544-e854-405a-abb0-a5021aacb771-kube-api-access-65ld6\") pod \"091d2544-e854-405a-abb0-a5021aacb771\" (UID: \"091d2544-e854-405a-abb0-a5021aacb771\") " Jul 22 16:54:22.556020 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.555992 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091d2544-e854-405a-abb0-a5021aacb771-kube-api-access-65ld6" (OuterVolumeSpecName: "kube-api-access-65ld6") pod "091d2544-e854-405a-abb0-a5021aacb771" (UID: "091d2544-e854-405a-abb0-a5021aacb771"). InnerVolumeSpecName "kube-api-access-65ld6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:54:22.593452 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.593410 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7694-140398118562352"] Jul 22 16:54:22.593887 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.593872 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091d2544-e854-405a-abb0-a5021aacb771" containerName="test-curl-7694-140398118560432" Jul 22 16:54:22.593957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.593889 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="091d2544-e854-405a-abb0-a5021aacb771" containerName="test-curl-7694-140398118560432" Jul 22 16:54:22.593957 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.593956 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="091d2544-e854-405a-abb0-a5021aacb771" containerName="test-curl-7694-140398118560432" Jul 22 16:54:22.597300 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.597279 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118562352" Jul 22 16:54:22.605810 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.605765 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-140398118562352"] Jul 22 16:54:22.654847 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.654811 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65ld6\" (UniqueName: \"kubernetes.io/projected/091d2544-e854-405a-abb0-a5021aacb771-kube-api-access-65ld6\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:54:22.755847 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.755802 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz268\" (UniqueName: \"kubernetes.io/projected/0ec7813a-e580-4299-bd8b-4ab836d858de-kube-api-access-vz268\") pod \"test-curl-7694-140398118562352\" (UID: \"0ec7813a-e580-4299-bd8b-4ab836d858de\") " pod="opendatahub/test-curl-7694-140398118562352" Jul 22 16:54:22.856999 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.856892 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz268\" (UniqueName: \"kubernetes.io/projected/0ec7813a-e580-4299-bd8b-4ab836d858de-kube-api-access-vz268\") pod \"test-curl-7694-140398118562352\" (UID: \"0ec7813a-e580-4299-bd8b-4ab836d858de\") " pod="opendatahub/test-curl-7694-140398118562352" Jul 22 16:54:22.867714 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.867672 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz268\" (UniqueName: \"kubernetes.io/projected/0ec7813a-e580-4299-bd8b-4ab836d858de-kube-api-access-vz268\") pod \"test-curl-7694-140398118562352\" (UID: \"0ec7813a-e580-4299-bd8b-4ab836d858de\") " pod="opendatahub/test-curl-7694-140398118562352" Jul 22 16:54:22.909884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:22.909840 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118562352" Jul 22 16:54:23.065100 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:23.065075 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-140398118562352"] Jul 22 16:54:23.065771 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:54:23.065749 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ec7813a_e580_4299_bd8b_4ab836d858de.slice/crio-776c2ee37624f328f9607edfc63ffdfe2292afdc9dc2e81551efb966dc563e06 WatchSource:0}: Error finding container 776c2ee37624f328f9607edfc63ffdfe2292afdc9dc2e81551efb966dc563e06: Status 404 returned error can't find the container with id 776c2ee37624f328f9607edfc63ffdfe2292afdc9dc2e81551efb966dc563e06 Jul 22 16:54:23.129830 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:23.129742 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091d2544-e854-405a-abb0-a5021aacb771" path="/var/lib/kubelet/pods/091d2544-e854-405a-abb0-a5021aacb771/volumes" Jul 22 16:54:23.236616 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:23.236565 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-140398118562352" event={"ID":"0ec7813a-e580-4299-bd8b-4ab836d858de","Type":"ContainerStarted","Data":"776c2ee37624f328f9607edfc63ffdfe2292afdc9dc2e81551efb966dc563e06"} Jul 22 16:54:23.237933 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:23.237906 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118560432" Jul 22 16:54:23.237933 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:23.237933 2564 scope.go:117] "RemoveContainer" containerID="6ce5672c480e0abc173fcf62762a6396d9ff8243d9a6dd1c564558dbfeead7a1" Jul 22 16:54:24.247118 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:24.247082 2564 generic.go:356] "Generic (PLEG): container finished" podID="0ec7813a-e580-4299-bd8b-4ab836d858de" containerID="26f7607c5c6dee51c78c740760dd44a357eb15dff4e4203d1c8f9d41ac04bca4" exitCode=0 Jul 22 16:54:24.247579 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:24.247179 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-140398118562352" event={"ID":"0ec7813a-e580-4299-bd8b-4ab836d858de","Type":"ContainerDied","Data":"26f7607c5c6dee51c78c740760dd44a357eb15dff4e4203d1c8f9d41ac04bca4"} Jul 22 16:54:25.404305 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:25.404275 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118562352" Jul 22 16:54:25.427059 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:25.427030 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7694-140398118562352_0ec7813a-e580-4299-bd8b-4ab836d858de/test-curl-7694-140398118562352/0.log" Jul 22 16:54:25.454616 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:25.454568 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7694-140398118562352"] Jul 22 16:54:25.457884 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:25.457852 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7694-140398118562352"] Jul 22 16:54:25.583744 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:25.583650 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz268\" (UniqueName: \"kubernetes.io/projected/0ec7813a-e580-4299-bd8b-4ab836d858de-kube-api-access-vz268\") pod \"0ec7813a-e580-4299-bd8b-4ab836d858de\" (UID: \"0ec7813a-e580-4299-bd8b-4ab836d858de\") " Jul 22 16:54:25.585792 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:25.585758 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ec7813a-e580-4299-bd8b-4ab836d858de-kube-api-access-vz268" (OuterVolumeSpecName: "kube-api-access-vz268") pod "0ec7813a-e580-4299-bd8b-4ab836d858de" (UID: "0ec7813a-e580-4299-bd8b-4ab836d858de"). InnerVolumeSpecName "kube-api-access-vz268". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:54:25.684568 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:25.684529 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz268\" (UniqueName: \"kubernetes.io/projected/0ec7813a-e580-4299-bd8b-4ab836d858de-kube-api-access-vz268\") on node \"ip-10-0-143-222.ec2.internal\" DevicePath \"\"" Jul 22 16:54:26.261779 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:26.261744 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-140398118562352" Jul 22 16:54:26.261967 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:26.261755 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="776c2ee37624f328f9607edfc63ffdfe2292afdc9dc2e81551efb966dc563e06" Jul 22 16:54:26.276224 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:54:26.276177 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7694-140398118562352\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="0ec7813a-e580-4299-bd8b-4ab836d858de" pod="opendatahub/test-curl-7694-140398118562352" Jul 22 16:54:27.129732 ip-10-0-143-222 kubenswrapper[2564]: E0722 16:54:27.129693 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7694-140398118562352\" is forbidden: User \"system:node:ip-10-0-143-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-222.ec2.internal' and this object" podUID="0ec7813a-e580-4299-bd8b-4ab836d858de" pod="opendatahub/test-curl-7694-140398118562352" Jul 22 16:54:27.131012 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:54:27.130988 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ec7813a-e580-4299-bd8b-4ab836d858de" path="/var/lib/kubelet/pods/0ec7813a-e580-4299-bd8b-4ab836d858de/volumes" Jul 22 16:55:22.061426 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:22.061398 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6455bbf649-9xppr_07fa8ebe-6ac0-4a0c-aa04-11e9fa708450/authorino/0.log" Jul 22 16:55:27.430886 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:27.430849 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lmx5s_e1ec9110-4bac-4048-bf96-059f1a209734/manager/0.log" Jul 22 16:55:27.549109 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:27.549082 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zngzz_5f36a9fc-12a1-4d97-baea-9255f80f5601/manager/0.log" Jul 22 16:55:27.777600 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:27.777571 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hmgdw_d14267c3-dc35-40e5-8fc0-6d6a2abd03e3/server/0.log" Jul 22 16:55:27.905150 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:27.905104 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7j6jk_bb0554b7-069b-4ee9-bd75-500d7283e11c/manager/0.log" Jul 22 16:55:28.149655 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:28.149556 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-587bf8b54-5bgrv_991db946-f941-4a34-94f4-3b72eca34a92/copy-manifests/0.log" Jul 22 16:55:28.162638 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:28.162597 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-587bf8b54-5bgrv_991db946-f941-4a34-94f4-3b72eca34a92/manager/0.log" Jul 22 16:55:28.635014 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:28.634988 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-56b457c467-xptbm_bcdb6efe-39a8-40f4-9630-478e51ada5b9/maas-api/0.log" Jul 22 16:55:28.989134 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:28.989100 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-9qpgh_57ce2cf3-cbbb-4494-b240-cf951620c0c6/postgres/0.log" Jul 22 16:55:29.798499 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:29.798470 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw_4bfb9882-5438-47e4-845e-855a85339297/extract/0.log" Jul 22 16:55:29.804972 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:29.804949 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw_4bfb9882-5438-47e4-845e-855a85339297/util/0.log" Jul 22 16:55:29.812415 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:29.812396 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw_4bfb9882-5438-47e4-845e-855a85339297/pull/0.log" Jul 22 16:55:29.927763 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:29.927747 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f_36e4e78f-4e20-49a2-ba3b-6e30a0e242e1/pull/0.log" Jul 22 16:55:29.934378 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:29.934361 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f_36e4e78f-4e20-49a2-ba3b-6e30a0e242e1/extract/0.log" Jul 22 16:55:29.940903 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:29.940881 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f_36e4e78f-4e20-49a2-ba3b-6e30a0e242e1/util/0.log" Jul 22 16:55:30.056168 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.056107 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k_6bf3a98c-b189-498c-a784-14b7a5c16199/util/0.log" Jul 22 16:55:30.063368 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.063350 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k_6bf3a98c-b189-498c-a784-14b7a5c16199/pull/0.log" Jul 22 16:55:30.070518 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.070495 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k_6bf3a98c-b189-498c-a784-14b7a5c16199/extract/0.log" Jul 22 16:55:30.195083 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.195060 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr_29b4435e-844a-4d1f-8755-b7409154f43c/util/0.log" Jul 22 16:55:30.202294 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.202276 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr_29b4435e-844a-4d1f-8755-b7409154f43c/pull/0.log" Jul 22 16:55:30.209148 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.209130 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr_29b4435e-844a-4d1f-8755-b7409154f43c/extract/0.log" Jul 22 16:55:30.329539 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.329467 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6455bbf649-9xppr_07fa8ebe-6ac0-4a0c-aa04-11e9fa708450/authorino/0.log" Jul 22 16:55:30.452201 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.452174 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cckvl_704d3a3e-ecfc-4dfe-8f2e-277902356b20/manager/0.log" Jul 22 16:55:30.565064 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.565032 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jcqff_5ea66736-bff9-4bec-9fe5-7b4842433781/manager/0.log" Jul 22 16:55:30.680234 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.680165 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nsd9z_d0d4aeb7-5c31-4e66-b082-a6f2160e7456/kuadrant-console-plugin/0.log" Jul 22 16:55:30.800765 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.800741 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-gjrpl_95bb3861-c7b6-44eb-8976-a2c62964e1ad/registry-server/0.log" Jul 22 16:55:30.927505 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:30.927481 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vkqh2_f9e55314-6e1f-4640-a6c4-3a44e4967b5f/manager/0.log" Jul 22 16:55:31.777761 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:31.777736 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b87bf6fb-7rljj_479e54eb-ec53-41a5-8b4c-2d7a5363bba0/kube-auth-proxy/0.log" Jul 22 16:55:32.025378 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:32.025352 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fwbxf_3127e95b-74b6-4ef2-ace4-63bba9953ffc/istio-proxy/1.log" Jul 22 16:55:32.149378 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:32.149305 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-x9p2f_f1b4e347-134e-485c-aabc-8d0aa3d358d3/payload-pre-processing/0.log" Jul 22 16:55:32.278058 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:32.278032 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-8ddlf_38a2b024-28a6-419a-ba25-9b19d12f6174/payload-processing/0.log" Jul 22 16:55:32.751447 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:32.751421 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-rbskq_ab10cd92-a9f3-4290-99f7-6c5b5fbfe419/main/0.log" Jul 22 16:55:32.875160 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:32.875134 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-jncjd_dca69f5a-b04e-433f-be7a-dc9d17e6e5b9/main/0.log" Jul 22 16:55:33.358015 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:33.357983 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-zjjrx_71488d8e-3c4d-43ae-b926-f44744f960b3/main/0.log" Jul 22 16:55:41.589673 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:41.589636 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7zwc4_7ce617c4-ea42-4943-8dde-54b9bfc24d0d/global-pull-secret-syncer/0.log" Jul 22 16:55:41.758877 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:41.758847 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vvbs2_e89a06fb-1480-471d-be5d-9758ece1a6f8/konnectivity-agent/0.log" Jul 22 16:55:41.834877 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:41.834850 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-222.ec2.internal_a761dd84e540e2dc089b8031677b8ec6/haproxy/0.log" Jul 22 16:55:46.371324 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.371291 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw_4bfb9882-5438-47e4-845e-855a85339297/extract/0.log" Jul 22 16:55:46.399672 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.399647 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw_4bfb9882-5438-47e4-845e-855a85339297/util/0.log" Jul 22 16:55:46.427558 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.427535 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598fpvw_4bfb9882-5438-47e4-845e-855a85339297/pull/0.log" Jul 22 16:55:46.461862 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.461842 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f_36e4e78f-4e20-49a2-ba3b-6e30a0e242e1/extract/0.log" Jul 22 16:55:46.484772 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.484753 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f_36e4e78f-4e20-49a2-ba3b-6e30a0e242e1/util/0.log" Jul 22 16:55:46.511478 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.511453 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p6k8f_36e4e78f-4e20-49a2-ba3b-6e30a0e242e1/pull/0.log" Jul 22 16:55:46.538501 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.538480 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k_6bf3a98c-b189-498c-a784-14b7a5c16199/extract/0.log" Jul 22 16:55:46.568307 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.568290 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k_6bf3a98c-b189-498c-a784-14b7a5c16199/util/0.log" Jul 22 16:55:46.601078 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.601059 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lv44k_6bf3a98c-b189-498c-a784-14b7a5c16199/pull/0.log" Jul 22 16:55:46.628478 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.628431 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr_29b4435e-844a-4d1f-8755-b7409154f43c/extract/0.log" Jul 22 16:55:46.650781 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.650762 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr_29b4435e-844a-4d1f-8755-b7409154f43c/util/0.log" Jul 22 16:55:46.674271 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:46.674253 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1s4jjr_29b4435e-844a-4d1f-8755-b7409154f43c/pull/0.log" Jul 22 16:55:47.005260 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:47.005231 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6455bbf649-9xppr_07fa8ebe-6ac0-4a0c-aa04-11e9fa708450/authorino/0.log" Jul 22 16:55:47.058125 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:47.058101 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cckvl_704d3a3e-ecfc-4dfe-8f2e-277902356b20/manager/0.log" Jul 22 16:55:47.101955 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:47.101931 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-jcqff_5ea66736-bff9-4bec-9fe5-7b4842433781/manager/0.log" Jul 22 16:55:47.128727 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:47.128699 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nsd9z_d0d4aeb7-5c31-4e66-b082-a6f2160e7456/kuadrant-console-plugin/0.log" Jul 22 16:55:47.179712 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:47.179688 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-gjrpl_95bb3861-c7b6-44eb-8976-a2c62964e1ad/registry-server/0.log" Jul 22 16:55:47.366063 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:47.365995 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vkqh2_f9e55314-6e1f-4640-a6c4-3a44e4967b5f/manager/0.log" Jul 22 16:55:49.317928 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:49.317899 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b93bec33-1718-4ba0-ba11-468de38e15e1/alertmanager/0.log" Jul 22 16:55:49.339358 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:49.339339 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b93bec33-1718-4ba0-ba11-468de38e15e1/config-reloader/0.log" Jul 22 16:55:49.360951 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:49.360910 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b93bec33-1718-4ba0-ba11-468de38e15e1/kube-rbac-proxy-web/0.log" Jul 22 16:55:49.384049 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:49.384032 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b93bec33-1718-4ba0-ba11-468de38e15e1/kube-rbac-proxy/0.log" Jul 22 16:55:49.405093 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:49.405074 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b93bec33-1718-4ba0-ba11-468de38e15e1/kube-rbac-proxy-metric/0.log" Jul 22 16:55:49.424938 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:49.424907 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b93bec33-1718-4ba0-ba11-468de38e15e1/prom-label-proxy/0.log" Jul 22 16:55:49.444880 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:49.444860 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b93bec33-1718-4ba0-ba11-468de38e15e1/init-config-reloader/0.log" Jul 22 16:55:49.729888 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:49.729859 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dwv8m_48066dc6-1e34-48dd-bfcf-9334dbbe2943/node-exporter/0.log" Jul 22 16:55:49.759324 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:49.759304 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dwv8m_48066dc6-1e34-48dd-bfcf-9334dbbe2943/kube-rbac-proxy/0.log" Jul 22 16:55:49.801490 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:49.801470 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dwv8m_48066dc6-1e34-48dd-bfcf-9334dbbe2943/init-textfile/0.log" Jul 22 16:55:50.280239 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:50.280205 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-rkdpm_a8636682-8baf-4b64-bd7f-29d7de537cfe/prometheus-operator-admission-webhook/0.log" Jul 22 16:55:50.378924 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:50.378897 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b558fdcbf-7r8l7_f991ce8c-0f02-4f1a-8209-a548098c3126/thanos-query/0.log" Jul 22 16:55:50.400666 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:50.400644 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b558fdcbf-7r8l7_f991ce8c-0f02-4f1a-8209-a548098c3126/kube-rbac-proxy-web/0.log" Jul 22 16:55:50.421923 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:50.421901 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b558fdcbf-7r8l7_f991ce8c-0f02-4f1a-8209-a548098c3126/kube-rbac-proxy/0.log" Jul 22 16:55:50.443586 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:50.443566 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b558fdcbf-7r8l7_f991ce8c-0f02-4f1a-8209-a548098c3126/prom-label-proxy/0.log" Jul 22 16:55:50.464535 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:50.464516 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b558fdcbf-7r8l7_f991ce8c-0f02-4f1a-8209-a548098c3126/kube-rbac-proxy-rules/0.log" Jul 22 16:55:50.493128 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:50.493109 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b558fdcbf-7r8l7_f991ce8c-0f02-4f1a-8209-a548098c3126/kube-rbac-proxy-metrics/0.log" Jul 22 16:55:51.697692 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:51.697668 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-wgvx5_1467496e-62de-4881-b8a9-9e090e9e0c1b/networking-console-plugin/0.log" Jul 22 16:55:51.956585 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:51.956499 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm"] Jul 22 16:55:51.957047 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:51.957029 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ec7813a-e580-4299-bd8b-4ab836d858de" containerName="test-curl-7694-140398118562352" Jul 22 16:55:51.957047 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:51.957048 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ec7813a-e580-4299-bd8b-4ab836d858de" containerName="test-curl-7694-140398118562352" Jul 22 16:55:51.957201 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:51.957150 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ec7813a-e580-4299-bd8b-4ab836d858de" containerName="test-curl-7694-140398118562352" Jul 22 16:55:51.960862 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:51.960841 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:51.963544 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:51.963525 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bzfkk\"/\"kube-root-ca.crt\"" Jul 22 16:55:51.964717 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:51.964702 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bzfkk\"/\"default-dockercfg-s9whn\"" Jul 22 16:55:51.964796 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:51.964703 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bzfkk\"/\"openshift-service-ca.crt\"" Jul 22 16:55:51.971662 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:51.971636 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm"] Jul 22 16:55:52.037459 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.037428 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-sys\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.037661 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.037485 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxzwq\" (UniqueName: \"kubernetes.io/projected/f7830306-99ba-4c34-a0b9-bce32aac1abd-kube-api-access-hxzwq\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.037661 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.037546 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-proc\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.037746 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.037705 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-lib-modules\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.037746 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.037742 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-podres\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.138800 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.138767 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-proc\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.138958 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.138821 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-lib-modules\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.138958 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.138853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-podres\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.138958 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.138886 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-sys\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.138958 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.138888 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-proc\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.138958 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.138949 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-sys\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.139127 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.138973 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxzwq\" (UniqueName: \"kubernetes.io/projected/f7830306-99ba-4c34-a0b9-bce32aac1abd-kube-api-access-hxzwq\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.139127 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.139007 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-lib-modules\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.139127 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.139019 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7830306-99ba-4c34-a0b9-bce32aac1abd-podres\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.148676 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.148619 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxzwq\" (UniqueName: \"kubernetes.io/projected/f7830306-99ba-4c34-a0b9-bce32aac1abd-kube-api-access-hxzwq\") pod \"perf-node-gather-daemonset-dgjdm\" (UID: \"f7830306-99ba-4c34-a0b9-bce32aac1abd\") " pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.271378 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.271348 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.407712 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.407683 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm"] Jul 22 16:55:52.408675 ip-10-0-143-222 kubenswrapper[2564]: W0722 16:55:52.408649 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7830306_99ba_4c34_a0b9_bce32aac1abd.slice/crio-73c4464f21f0a168a23c345f071a3f034025ba9358de7e2054975dd876410c77 WatchSource:0}: Error finding container 73c4464f21f0a168a23c345f071a3f034025ba9358de7e2054975dd876410c77: Status 404 returned error can't find the container with id 73c4464f21f0a168a23c345f071a3f034025ba9358de7e2054975dd876410c77 Jul 22 16:55:52.766973 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.766930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" event={"ID":"f7830306-99ba-4c34-a0b9-bce32aac1abd","Type":"ContainerStarted","Data":"7e3fbf8918bc6b48362cc3e3815e254ba92f9b089aba8000b088be4074cabbb8"} Jul 22 16:55:52.767419 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.766986 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:55:52.767419 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.767008 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" event={"ID":"f7830306-99ba-4c34-a0b9-bce32aac1abd","Type":"ContainerStarted","Data":"73c4464f21f0a168a23c345f071a3f034025ba9358de7e2054975dd876410c77"} Jul 22 16:55:52.785926 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.785873 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" podStartSLOduration=1.785858029 podStartE2EDuration="1.785858029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:55:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:55:52.782846541 +0000 UTC m=+2916.178388375" watchObservedRunningTime="2026-07-22 16:55:52.785858029 +0000 UTC m=+2916.181399862" Jul 22 16:55:52.882327 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.882299 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-ff8c64968-68mtq_fa87156c-4e62-47e8-a6fb-7acbf0724dfc/console/0.log" Jul 22 16:55:52.914867 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:52.914840 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-rqbx8_aa4d869d-91fb-45e6-9466-f0bbd0ae957c/download-server/0.log" Jul 22 16:55:54.567176 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:54.567147 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nhdrz_7282bf8f-d9c6-4f64-8e4f-ab36bef8369d/dns/0.log" Jul 22 16:55:54.594213 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:54.594190 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nhdrz_7282bf8f-d9c6-4f64-8e4f-ab36bef8369d/kube-rbac-proxy/0.log" Jul 22 16:55:54.668506 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:54.668482 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hb82z_e98f2ec9-31ce-45fd-a11f-9bb46626c62b/dns-node-resolver/0.log" Jul 22 16:55:55.270869 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:55.270839 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-77fb4c4bf8-xdzrw_7db24c11-eba3-43d4-8491-7f8897046261/registry/0.log" Jul 22 16:55:55.316364 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:55.316337 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nrhcg_d64c7c2f-b2b4-4738-88dd-e56aeeadd228/node-ca/0.log" Jul 22 16:55:56.409666 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:56.409618 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b87bf6fb-7rljj_479e54eb-ec53-41a5-8b4c-2d7a5363bba0/kube-auth-proxy/0.log" Jul 22 16:55:56.461862 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:56.461837 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fwbxf_3127e95b-74b6-4ef2-ace4-63bba9953ffc/istio-proxy/0.log" Jul 22 16:55:56.486057 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:56.486031 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fwbxf_3127e95b-74b6-4ef2-ace4-63bba9953ffc/istio-proxy/1.log" Jul 22 16:55:56.515283 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:56.515263 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-x9p2f_f1b4e347-134e-485c-aabc-8d0aa3d358d3/payload-pre-processing/0.log" Jul 22 16:55:56.545406 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:56.545383 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-8ddlf_38a2b024-28a6-419a-ba25-9b19d12f6174/payload-processing/0.log" Jul 22 16:55:57.339750 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:57.339714 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w4dkw_58b9ab02-b98e-466f-b11c-2600c9e47261/serve-healthcheck-canary/0.log" Jul 22 16:55:57.900288 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:57.900262 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j4g57_201d6928-9fae-43b8-b0a4-d1ce64c9d9c7/kube-rbac-proxy/0.log" Jul 22 16:55:57.919743 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:57.919717 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j4g57_201d6928-9fae-43b8-b0a4-d1ce64c9d9c7/exporter/0.log" Jul 22 16:55:57.941491 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:57.941473 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j4g57_201d6928-9fae-43b8-b0a4-d1ce64c9d9c7/extractor/0.log" Jul 22 16:55:58.783183 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:55:58.783157 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bzfkk/perf-node-gather-daemonset-dgjdm" Jul 22 16:56:00.251308 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:00.251282 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lmx5s_e1ec9110-4bac-4048-bf96-059f1a209734/manager/0.log" Jul 22 16:56:00.431211 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:00.431178 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zngzz_5f36a9fc-12a1-4d97-baea-9255f80f5601/manager/0.log" Jul 22 16:56:00.625701 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:00.625675 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hmgdw_d14267c3-dc35-40e5-8fc0-6d6a2abd03e3/server/0.log" Jul 22 16:56:00.719344 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:00.719311 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7j6jk_bb0554b7-069b-4ee9-bd75-500d7283e11c/manager/0.log" Jul 22 16:56:00.878222 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:00.878196 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-587bf8b54-5bgrv_991db946-f941-4a34-94f4-3b72eca34a92/manager/0.log" Jul 22 16:56:00.908434 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:00.908404 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-587bf8b54-5bgrv_991db946-f941-4a34-94f4-3b72eca34a92/copy-manifests/0.log" Jul 22 16:56:02.209550 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:02.209527 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b87bf6fb-7rljj_479e54eb-ec53-41a5-8b4c-2d7a5363bba0/kube-auth-proxy/0.log" Jul 22 16:56:02.249938 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:02.249917 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fwbxf_3127e95b-74b6-4ef2-ace4-63bba9953ffc/istio-proxy/0.log" Jul 22 16:56:02.279206 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:02.279182 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fwbxf_3127e95b-74b6-4ef2-ace4-63bba9953ffc/istio-proxy/1.log" Jul 22 16:56:02.301316 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:02.301291 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-x9p2f_f1b4e347-134e-485c-aabc-8d0aa3d358d3/payload-pre-processing/0.log" Jul 22 16:56:02.319827 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:02.319808 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-8ddlf_38a2b024-28a6-419a-ba25-9b19d12f6174/payload-processing/0.log" Jul 22 16:56:03.630479 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:03.630439 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b87bf6fb-7rljj_479e54eb-ec53-41a5-8b4c-2d7a5363bba0/kube-auth-proxy/0.log" Jul 22 16:56:03.668208 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:03.668182 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fwbxf_3127e95b-74b6-4ef2-ace4-63bba9953ffc/istio-proxy/0.log" Jul 22 16:56:03.697055 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:03.697029 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fwbxf_3127e95b-74b6-4ef2-ace4-63bba9953ffc/istio-proxy/1.log" Jul 22 16:56:03.730376 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:03.730348 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-x9p2f_f1b4e347-134e-485c-aabc-8d0aa3d358d3/payload-pre-processing/0.log" Jul 22 16:56:03.752222 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:03.752198 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-8ddlf_38a2b024-28a6-419a-ba25-9b19d12f6174/payload-processing/0.log" Jul 22 16:56:08.716525 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:08.716499 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4tmkn_313873ed-89d3-43ad-b540-633cb5cbe79a/kube-multus/0.log" Jul 22 16:56:08.741387 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:08.741358 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p8jnb_91b8bd81-42f0-4480-9f6f-cdac9397a125/kube-multus-additional-cni-plugins/0.log" Jul 22 16:56:08.764692 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:08.764661 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p8jnb_91b8bd81-42f0-4480-9f6f-cdac9397a125/egress-router-binary-copy/0.log" Jul 22 16:56:08.785109 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:08.785091 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p8jnb_91b8bd81-42f0-4480-9f6f-cdac9397a125/cni-plugins/0.log" Jul 22 16:56:08.804592 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:08.804571 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p8jnb_91b8bd81-42f0-4480-9f6f-cdac9397a125/bond-cni-plugin/0.log" Jul 22 16:56:08.825904 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:08.825884 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p8jnb_91b8bd81-42f0-4480-9f6f-cdac9397a125/routeoverride-cni/0.log" Jul 22 16:56:08.846456 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:08.846433 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p8jnb_91b8bd81-42f0-4480-9f6f-cdac9397a125/whereabouts-cni-bincopy/0.log" Jul 22 16:56:08.869130 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:08.869112 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p8jnb_91b8bd81-42f0-4480-9f6f-cdac9397a125/whereabouts-cni/0.log" Jul 22 16:56:09.383515 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:09.383492 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lfc4g_02186816-0aad-4a17-943b-e09d6dca7a03/network-metrics-daemon/0.log" Jul 22 16:56:09.401825 ip-10-0-143-222 kubenswrapper[2564]: I0722 16:56:09.401799 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lfc4g_02186816-0aad-4a17-943b-e09d6dca7a03/kube-rbac-proxy/0.log"