Jul 22 16:40:12.960618 ip-10-0-132-57 systemd[1]: Starting Kubernetes Kubelet... Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.405549 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413224 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413236 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413239 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413242 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413245 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413248 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413251 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:40:13.688239 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413254 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413256 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413259 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413262 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413265 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413267 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413269 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413274 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413276 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413279 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413282 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413285 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413287 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413290 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413293 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413296 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413299 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413301 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413304 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413307 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:40:13.699196 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413310 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:40:13.492583 ip-10-0-132-57 systemd[1]: Started Kubernetes Kubelet. Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413313 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413316 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413320 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413324 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413329 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413332 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413335 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413338 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413340 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413343 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413348 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413351 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413354 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413356 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413359 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413377 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413381 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413384 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:40:13.701668 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413388 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413393 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413397 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413400 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413403 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413405 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413408 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413410 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413413 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413415 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413418 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413420 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413423 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413426 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413428 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413431 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413434 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413437 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413439 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:40:13.703408 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413442 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413444 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413447 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413450 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413452 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413455 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413458 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413461 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413464 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413467 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413469 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413472 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413475 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413478 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413481 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413484 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413487 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413489 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413492 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413495 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:40:13.704376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413497 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413500 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413504 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413506 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413509 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413511 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413514 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413516 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413519 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413522 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413524 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413527 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413530 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.413532 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415905 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415913 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415917 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415920 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415937 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415941 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415944 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:40:13.706445 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415947 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415950 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415953 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415956 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415959 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415962 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415964 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415967 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415970 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415973 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415977 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415981 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415984 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415987 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415989 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415992 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415995 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.415998 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416001 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:40:13.708672 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416003 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416006 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416008 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416011 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416014 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416016 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416019 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416022 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416025 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416027 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416030 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416033 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416035 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416038 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416041 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416044 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416046 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416049 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416051 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416054 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:40:13.710927 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416056 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416062 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416064 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416067 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416070 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416072 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416075 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416077 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416080 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416082 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416086 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416089 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416092 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416094 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416097 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416099 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416102 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416105 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416107 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:40:13.711994 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416110 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416114 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416117 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416121 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416123 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416126 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416129 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416132 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416135 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416137 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416139 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416142 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416144 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416150 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416152 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416157 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416160 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416162 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416164 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416167 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:40:13.713657 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416169 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416172 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416175 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416177 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416180 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416182 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416185 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416189 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416191 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416194 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416198 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416201 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416204 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416207 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416587 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416592 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416595 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416598 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416600 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:40:13.715376 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416603 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416606 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416608 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416611 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416613 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416616 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416620 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416622 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416631 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416634 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416637 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416640 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416642 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416645 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416647 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416650 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416652 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416655 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416657 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416660 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:40:13.716761 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416662 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416665 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416668 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416671 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416673 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416675 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416678 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416680 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416683 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416685 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416688 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416690 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416692 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416695 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416697 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416700 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416702 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416705 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416709 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416711 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:40:13.717775 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416716 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416720 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416723 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416726 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416729 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416731 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416734 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416736 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416739 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416741 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416744 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416746 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416749 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416751 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416754 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416756 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416759 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416761 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416765 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416767 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:40:13.718863 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416770 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416772 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416775 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416777 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416780 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416782 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416785 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416787 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416789 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416792 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416795 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416799 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416804 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416808 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416812 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416815 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416818 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416821 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416823 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:40:13.719716 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416826 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416828 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416831 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416833 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416836 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416838 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416841 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416844 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416846 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416850 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416853 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416856 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416859 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416862 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.416864 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416932 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416939 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416944 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416948 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416952 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416955 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 16:40:13.720577 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416959 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416963 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416967 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416970 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416974 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416978 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416981 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416985 2581 flags.go:64] FLAG: --cgroup-root="" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416988 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416991 2581 flags.go:64] FLAG: --client-ca-file="" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416994 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.416997 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417001 2581 flags.go:64] FLAG: --cluster-domain="" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417004 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417007 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417010 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417013 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417017 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417020 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417023 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417026 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417029 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417032 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417035 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 16:40:13.721644 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417038 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417040 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417044 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417047 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417050 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417052 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417055 2581 flags.go:64] FLAG: --enable-server="true" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417058 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417062 2581 flags.go:64] FLAG: --event-burst="100" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417065 2581 flags.go:64] FLAG: --event-qps="50" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417070 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417073 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417077 2581 flags.go:64] FLAG: --eviction-hard="" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417080 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417083 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417086 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417089 2581 flags.go:64] FLAG: --eviction-soft="" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417092 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417095 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417098 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417101 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417104 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417106 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417109 2581 flags.go:64] FLAG: --feature-gates="" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417113 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 16:40:13.722727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417116 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417119 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417121 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417124 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417128 2581 flags.go:64] FLAG: --help="false" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417130 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417133 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417136 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417139 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417142 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417146 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417148 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417151 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417154 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417157 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417159 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417162 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417166 2581 flags.go:64] FLAG: --kube-reserved="" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417169 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417173 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417176 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417179 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417182 2581 flags.go:64] FLAG: --lock-file="" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417185 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 16:40:13.723804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417188 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417191 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417196 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417199 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417202 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417204 2581 flags.go:64] FLAG: --logging-format="text" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417207 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417210 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417213 2581 flags.go:64] FLAG: --manifest-url="" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417216 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417220 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417223 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417227 2581 flags.go:64] FLAG: --max-pods="110" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417229 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417232 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417235 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417238 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417241 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417243 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417246 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417253 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417256 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417259 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 16:40:13.724875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417262 2581 flags.go:64] FLAG: --pod-cidr="" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417265 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417270 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417274 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417279 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417282 2581 flags.go:64] FLAG: --port="10250" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417285 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417288 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-041ee2368d0e5d479" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417291 2581 flags.go:64] FLAG: --qos-reserved="" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417294 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417298 2581 flags.go:64] FLAG: --register-node="true" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417301 2581 flags.go:64] FLAG: --register-with-taints="" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417304 2581 flags.go:64] FLAG: --registry-burst="10" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417307 2581 flags.go:64] FLAG: --registry-qps="5" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417310 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417312 2581 flags.go:64] FLAG: --reserved-memory="" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417316 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417319 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417322 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417325 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417328 2581 flags.go:64] FLAG: --runonce="false" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417330 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417333 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417336 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417339 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 16:40:13.728028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417342 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417345 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417348 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417351 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417354 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417356 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417359 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417385 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417389 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417392 2581 flags.go:64] FLAG: --system-cgroups="" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417396 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417403 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417406 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417409 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417413 2581 flags.go:64] FLAG: --tls-min-version="" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417416 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417419 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417421 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417425 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417428 2581 flags.go:64] FLAG: --v="2" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417432 2581 flags.go:64] FLAG: --version="false" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417435 2581 flags.go:64] FLAG: --vmodule="" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417439 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.417442 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417560 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:40:13.729120 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417564 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417568 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417571 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417575 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417579 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417582 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417585 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417588 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417591 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417593 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417596 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417598 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417601 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417603 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417606 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417608 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417611 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417616 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417620 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417622 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:40:13.736286 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417625 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417627 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417630 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417633 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417635 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417638 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417641 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417643 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417646 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417648 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417651 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417654 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417656 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417659 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417661 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417664 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417666 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417669 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417672 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417674 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:40:13.737152 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417677 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417679 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417682 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417684 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417687 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417689 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417692 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417694 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417697 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417702 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417706 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417709 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417711 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417714 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417716 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417719 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417723 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417726 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417730 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417733 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:40:13.738019 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417736 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417740 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417743 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417746 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417749 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417752 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417754 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417757 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417760 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417762 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417765 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417768 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417770 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417773 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417775 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417778 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417780 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417783 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417785 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:40:13.738883 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417788 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417790 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417794 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417797 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417800 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417802 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417805 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417807 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417810 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417812 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417815 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417817 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417820 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417828 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417831 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417834 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417837 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417839 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:40:13.739754 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.417842 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.418612 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.426708 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.426723 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.426742 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.426748 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426801 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426805 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426809 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426812 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426815 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426818 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426821 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426824 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:40:13.740620 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426826 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426829 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426831 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426834 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426837 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426840 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426842 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426845 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426847 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426850 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426853 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426856 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426859 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426861 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426864 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426866 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426869 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426872 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426875 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426877 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426880 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:40:13.741295 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426882 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426885 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426888 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426890 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426898 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426902 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426904 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426907 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426910 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426913 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426916 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426918 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426921 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426923 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426928 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426931 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426933 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426936 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426940 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:40:13.742179 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426944 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426947 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426949 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426952 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426954 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426957 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426959 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426962 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426964 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426967 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426969 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426972 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426974 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426977 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426980 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426982 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426985 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426989 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426992 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:40:13.743058 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426995 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.426997 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427000 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427002 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427005 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427008 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427010 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427015 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427019 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427023 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427025 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427028 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427032 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427034 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427037 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427040 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427042 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427045 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427048 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427050 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:40:13.743926 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427053 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427055 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427058 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427061 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427063 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427065 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427077 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427080 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427082 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427085 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427089 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427092 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.427097 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427207 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427212 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:40:13.744794 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427216 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427218 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427222 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427227 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427230 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427235 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427238 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427241 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427244 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427247 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427250 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427252 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427255 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427258 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427261 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427263 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427266 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427268 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427271 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:40:13.745432 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427273 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427276 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427279 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427281 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427284 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427286 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427289 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427291 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427294 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427297 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427300 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427303 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427305 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427308 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427310 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427312 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427315 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427318 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427321 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427324 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:40:13.746308 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427326 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427329 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427332 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427335 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427338 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427341 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427343 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427347 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427350 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427353 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427356 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427359 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427361 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427383 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427386 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427388 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427391 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427393 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427396 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:40:13.747186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427399 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427401 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427405 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427408 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427411 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427413 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427416 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427418 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427421 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427423 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427426 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427430 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427432 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427435 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427438 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427440 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427443 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427445 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427449 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427451 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:40:13.748094 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427454 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427456 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427459 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427461 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427464 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427466 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427469 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427471 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427475 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427478 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427481 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427484 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427487 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427489 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427493 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427496 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427498 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427501 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:40:13.910697 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:13.427504 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.427509 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.427606 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.430948 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.432660 2581 server.go:1019] "Starting client certificate rotation" Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.432756 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.432793 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.435484 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.438568 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.452574 2581 log.go:25] "Validated CRI v1 runtime API" Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.456082 2581 log.go:25] "Validated CRI v1 image API" Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.458967 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 16:40:13.911712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.461659 2581 fs.go:135] Filesystem UUIDs: map[3343eddc-ae2d-44d4-9c0e-c2799b228956:/dev/nvme0n1p4 5e420437-2563-4c7f-9675-75f696ecdb43:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 22 16:40:13.912347 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.461673 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 16:40:13.912347 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.465480 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.467322 2581 manager.go:216] Machine: {Timestamp:2026-07-22 16:40:13.466583936 +0000 UTC m=+0.377241100 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100185 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c409534962a8f3ec6cbf51f98dd55 SystemUUID:ec2c4095-3496-2a8f-3ec6-cbf51f98dd55 BootID:909444ba-682e-4fed-86cb-c270179ec689 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b4:03:be:a9:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b4:03:be:a9:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:c6:09:a3:3a:30 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.467425 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.467493 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.470355 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.470392 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-57.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.471219 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.471228 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.471236 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.473003 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.474766 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.479508 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.479627 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.479655 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.479667 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.479684 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.479698 2581 kubelet.go:398] "Adding apiserver pod source" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.479709 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 16:40:13.912772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.480066 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.480084 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.483145 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.485531 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.485558 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.486982 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.486996 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.487002 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.487008 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.487016 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.487022 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.487030 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.487036 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.487044 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.487049 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.487058 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.487063 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.488788 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.488800 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.491304 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.491386 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.491712 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-57.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.491846 2581 server.go:1268] "Started kubelet" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.491978 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.492046 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.492601 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.493634 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.494178 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.497440 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.503178 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.503925 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.504036 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.504644 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.504645 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.504668 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.504746 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.504752 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.507190 2581 factory.go:55] Registering systemd factory Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.507203 2581 factory.go:223] Registration of the systemd container factory successfully Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.507396 2581 factory.go:153] Registering CRI-O factory Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.507408 2581 factory.go:223] Registration of the crio container factory successfully Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.507467 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.507494 2581 factory.go:103] Registering Raw factory Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.507507 2581 manager.go:1196] Started watching for new ooms in manager Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.507934 2581 manager.go:349] Starting recovery of all containers Jul 22 16:40:13.915923 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.509319 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.509430 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.509630 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-57.ec2.internal.18c4aa21912c77ec default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-57.ec2.internal,UID:ip-10-0-132-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-57.ec2.internal,},FirstTimestamp:2026-07-22 16:40:13.491812332 +0000 UTC m=+0.402469494,LastTimestamp:2026-07-22 16:40:13.491812332 +0000 UTC m=+0.402469494,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-57.ec2.internal,}" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.519110 2581 manager.go:354] Recovery completed Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.525331 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.527084 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.527116 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.527130 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.527657 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.527666 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.527679 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.527708 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.528292 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.528309 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.528318 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.529018 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-57.ec2.internal.18c4aa219346e51f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-57.ec2.internal,UID:ip-10-0-132-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-57.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-57.ec2.internal,},FirstTimestamp:2026-07-22 16:40:13.527098655 +0000 UTC m=+0.437755822,LastTimestamp:2026-07-22 16:40:13.527098655 +0000 UTC m=+0.437755822,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-57.ec2.internal,}" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.529817 2581 policy_none.go:49] "None policy: Start" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.529829 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.529839 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.532010 2581 policy_none.go:47] "Start" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.538046 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4s2bq" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.542430 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-57.ec2.internal.18c4aa2193474517 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-57.ec2.internal,UID:ip-10-0-132-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-57.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-57.ec2.internal,},FirstTimestamp:2026-07-22 16:40:13.527123223 +0000 UTC m=+0.437780386,LastTimestamp:2026-07-22 16:40:13.527123223 +0000 UTC m=+0.437780386,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-57.ec2.internal,}" Jul 22 16:40:13.917873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.545501 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4s2bq" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.552723 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-57.ec2.internal.18c4aa219347746b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-57.ec2.internal,UID:ip-10-0-132-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-57.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-57.ec2.internal,},FirstTimestamp:2026-07-22 16:40:13.527135339 +0000 UTC m=+0.437792503,LastTimestamp:2026-07-22 16:40:13.527135339 +0000 UTC m=+0.437792503,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-57.ec2.internal,}" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.571898 2581 manager.go:337] "Starting Device Plugin manager" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.571924 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.571932 2581 server.go:87] "Starting device plugin registration server" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.572133 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.572146 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.572232 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.572301 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.572308 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.573403 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.573451 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.611148 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.612241 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.612285 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.612308 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.612345 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.614814 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.672326 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.673957 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.673984 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.673993 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.674014 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.685119 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.685135 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-57.ec2.internal\": node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.714854 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.754502 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal"] Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.754555 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.755298 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.755321 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.755330 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.757546 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.757654 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.757681 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.758186 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.758212 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.758186 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:40:13.922025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.758257 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.758223 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.758270 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.760433 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.760460 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.761085 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.761104 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.761116 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.777864 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-57.ec2.internal\" not found" node="ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.785533 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-57.ec2.internal\" not found" node="ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.806812 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a836bcb889abc73302d1336d89ed5761-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"a836bcb889abc73302d1336d89ed5761\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.806835 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a836bcb889abc73302d1336d89ed5761-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"a836bcb889abc73302d1336d89ed5761\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.806853 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb8eed94bb9bf5d97c7ef83964eedb6a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-57.ec2.internal\" (UID: \"fb8eed94bb9bf5d97c7ef83964eedb6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.815017 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.907741 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a836bcb889abc73302d1336d89ed5761-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"a836bcb889abc73302d1336d89ed5761\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.907767 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a836bcb889abc73302d1336d89ed5761-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"a836bcb889abc73302d1336d89ed5761\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.907788 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb8eed94bb9bf5d97c7ef83964eedb6a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-57.ec2.internal\" (UID: \"fb8eed94bb9bf5d97c7ef83964eedb6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.907829 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb8eed94bb9bf5d97c7ef83964eedb6a-config\") pod \"kube-apiserver-proxy-ip-10-0-132-57.ec2.internal\" (UID: \"fb8eed94bb9bf5d97c7ef83964eedb6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.907871 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a836bcb889abc73302d1336d89ed5761-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"a836bcb889abc73302d1336d89ed5761\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:13.907899 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a836bcb889abc73302d1336d89ed5761-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"a836bcb889abc73302d1336d89ed5761\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Jul 22 16:40:13.923012 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:13.915841 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:14.092513 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:14.016385 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:14.092513 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.082543 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Jul 22 16:40:14.092513 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.088246 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Jul 22 16:40:14.242915 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:14.116789 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:14.242915 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:14.217256 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:14.396511 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:14.317797 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:14.547411 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:14.418231 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:14.547411 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.431389 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 16:40:14.547411 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.431527 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 16:40:14.547411 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:14.518955 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:14.555281 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.552143 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 16:35:14 +0000 UTC" deadline="2028-02-17 02:21:26.723675043 +0000 UTC" Jul 22 16:40:14.555281 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.552173 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13785h41m12.171506098s" Jul 22 16:40:14.689485 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.562038 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:40:14.689485 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:14.619081 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:14.689485 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.676129 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:40:14.689485 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.676147 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:40:14.689485 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.677506 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:40:14.689485 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.677543 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:40:14.689485 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.677559 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:40:14.849186 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:14.719952 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Jul 22 16:40:14.849186 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.739782 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:40:14.849186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:14.748786 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda836bcb889abc73302d1336d89ed5761.slice/crio-284d8d938bbef5e3352a950a131966b11415d32804f27964b54f9ddd5bf6e7ca WatchSource:0}: Error finding container 284d8d938bbef5e3352a950a131966b11415d32804f27964b54f9ddd5bf6e7ca: Status 404 returned error can't find the container with id 284d8d938bbef5e3352a950a131966b11415d32804f27964b54f9ddd5bf6e7ca Jul 22 16:40:14.849186 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.760738 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:40:14.849186 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:14.767622 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8eed94bb9bf5d97c7ef83964eedb6a.slice/crio-d3c51361daf468acbb4015059aa7e93cae97836f5e6472d48384f9bbf408b88e WatchSource:0}: Error finding container d3c51361daf468acbb4015059aa7e93cae97836f5e6472d48384f9bbf408b88e: Status 404 returned error can't find the container with id d3c51361daf468acbb4015059aa7e93cae97836f5e6472d48384f9bbf408b88e Jul 22 16:40:14.849186 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.804530 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Jul 22 16:40:14.849186 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.813813 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 16:40:14.849186 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.814717 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Jul 22 16:40:14.849186 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:14.828291 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 16:40:15.489899 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.481099 2581 apiserver.go:52] "Watching apiserver" Jul 22 16:40:15.489899 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.488131 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 16:40:15.489899 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.488543 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal","openshift-multus/network-metrics-daemon-ld4x4","openshift-network-diagnostics/network-check-target-jfkr8","kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal","openshift-dns/node-resolver-7tdk9","openshift-multus/multus-additional-cni-plugins-8xzv4","openshift-multus/multus-qj8th","openshift-network-operator/iptables-alerter-x9j2s","openshift-ovn-kubernetes/ovnkube-node-dsxzr","kube-system/global-pull-secret-syncer-dmtxl","kube-system/konnectivity-agent-t6d5r","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6","openshift-cluster-node-tuning-operator/tuned-j2c2g","openshift-image-registry/node-ca-nxfwf"] Jul 22 16:40:15.500792 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.491474 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:15.500792 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.491560 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:15.500792 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.494619 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.500792 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.496951 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:15.500792 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.497026 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:15.500792 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.497313 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:40:15.500792 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.497356 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c545c\"" Jul 22 16:40:15.500792 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.497688 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 16:40:15.500792 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.499323 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.503356 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.502175 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f6hjb\"" Jul 22 16:40:15.503356 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.502422 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 16:40:15.503356 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.502643 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 16:40:15.503356 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.502879 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 16:40:15.503356 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.503106 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.505457 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.503829 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 16:40:15.507287 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.505969 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 16:40:15.507287 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.506606 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xh5fr\"" Jul 22 16:40:15.507287 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.506789 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 16:40:15.507287 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.507029 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.509378 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.508032 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.511501 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.509574 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.511501 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.510749 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 16:40:15.511501 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.510948 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 16:40:15.511501 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.511141 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 16:40:15.511501 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.511328 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.511518 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.514415 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fx4tc\"" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.514850 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8zlgc\"" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515100 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7xfxb\"" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515411 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515599 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-sysctl-conf\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515643 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b77686d0-8cd0-4908-98fb-f34ff9a63c79-serviceca\") pod \"node-ca-nxfwf\" (UID: \"b77686d0-8cd0-4908-98fb-f34ff9a63c79\") " pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515678 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nb2g\" (UniqueName: \"kubernetes.io/projected/b77686d0-8cd0-4908-98fb-f34ff9a63c79-kube-api-access-8nb2g\") pod \"node-ca-nxfwf\" (UID: \"b77686d0-8cd0-4908-98fb-f34ff9a63c79\") " pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515714 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515749 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-run-ovn-kubernetes\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515784 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515816 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-system-cni-dir\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-kubelet-config\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515882 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515896 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.515938 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-sysconfig\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516008 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516033 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-lib-modules\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516068 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-host\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.583053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516110 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-tuned\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516143 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-tmp\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516176 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1de09de-782e-4b78-a68d-cbf1078cbc90-tmp-dir\") pod \"node-resolver-7tdk9\" (UID: \"d1de09de-782e-4b78-a68d-cbf1078cbc90\") " pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516210 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-systemd\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516232 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516263 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-sys\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516289 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr959\" (UniqueName: \"kubernetes.io/projected/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-kube-api-access-kr959\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516321 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-os-release\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516351 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-kubelet\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516379 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516378 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516384 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-run-systemd\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516605 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516630 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-var-lib-openvswitch\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516673 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-var-lib-kubelet\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516709 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516759 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhsdl\" (UniqueName: \"kubernetes.io/projected/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-kube-api-access-dhsdl\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516875 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.516954 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.517749 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.584941 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.516801 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-cni-bin\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.518814 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-cni-netd\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.518868 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwtls\" (UniqueName: \"kubernetes.io/projected/31962401-01b7-4b65-aa86-e64db7ad9af4-kube-api-access-lwtls\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.518905 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-kubernetes\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.518936 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b77686d0-8cd0-4908-98fb-f34ff9a63c79-host\") pod \"node-ca-nxfwf\" (UID: \"b77686d0-8cd0-4908-98fb-f34ff9a63c79\") " pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.518965 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519028 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-systemd-units\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519069 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-slash\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519101 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-run-openvswitch\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519138 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-node-log\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519169 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-log-socket\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519193 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmkwh\" (UniqueName: \"kubernetes.io/projected/d1de09de-782e-4b78-a68d-cbf1078cbc90-kube-api-access-pmkwh\") pod \"node-resolver-7tdk9\" (UID: \"d1de09de-782e-4b78-a68d-cbf1078cbc90\") " pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-dbus\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519313 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-modprobe-d\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519344 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-run-netns\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519428 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-etc-openvswitch\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.588398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519458 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-run-ovn\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519489 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/31962401-01b7-4b65-aa86-e64db7ad9af4-ovn-node-metrics-cert\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519517 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-sysctl-d\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519558 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-run\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519589 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1de09de-782e-4b78-a68d-cbf1078cbc90-hosts-file\") pod \"node-resolver-7tdk9\" (UID: \"d1de09de-782e-4b78-a68d-cbf1078cbc90\") " pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519627 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnqh8\" (UniqueName: \"kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8\") pod \"network-check-target-jfkr8\" (UID: \"e1f24e3f-ab56-4fa0-951b-45c19ed83baf\") " pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519696 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/31962401-01b7-4b65-aa86-e64db7ad9af4-ovnkube-config\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519749 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-cnibin\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519807 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/31962401-01b7-4b65-aa86-e64db7ad9af4-env-overrides\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.519839 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/31962401-01b7-4b65-aa86-e64db7ad9af4-ovnkube-script-lib\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.520923 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.520935 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.521392 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.522115 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rk5gw\"" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.522279 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.522347 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.523424 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.523526 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.523910 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.524002 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pkb2j\"" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.526206 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.526474 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.526589 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 16:40:15.589801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.526476 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jz65p\"" Jul 22 16:40:15.593959 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.546270 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6d8j5" Jul 22 16:40:15.593959 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.556278 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6d8j5" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.607915 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620474 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-system-cni-dir\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620503 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-kubelet-config\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620526 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-sysconfig\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620568 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-kubelet\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620593 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n27tg\" (UniqueName: \"kubernetes.io/projected/7832f593-a763-4b0f-be98-f5c73bec2465-kube-api-access-n27tg\") pod \"iptables-alerter-x9j2s\" (UID: \"7832f593-a763-4b0f-be98-f5c73bec2465\") " pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620615 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-system-cni-dir\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620635 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-os-release\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620655 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1de09de-782e-4b78-a68d-cbf1078cbc90-tmp-dir\") pod \"node-resolver-7tdk9\" (UID: \"d1de09de-782e-4b78-a68d-cbf1078cbc90\") " pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620673 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-sys\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.620694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr959\" (UniqueName: \"kubernetes.io/projected/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-kube-api-access-kr959\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.624185 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-system-cni-dir\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.624259 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-kubelet-config\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.624362 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.624449 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret podName:e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a nodeName:}" failed. No retries permitted until 2026-07-22 16:40:16.124406363 +0000 UTC m=+3.035063520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret") pod "global-pull-secret-syncer-dmtxl" (UID: "e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:15.694728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.624551 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-kubelet\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.624617 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-sysconfig\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.624666 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-sys\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.624835 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1de09de-782e-4b78-a68d-cbf1078cbc90-tmp-dir\") pod \"node-resolver-7tdk9\" (UID: \"d1de09de-782e-4b78-a68d-cbf1078cbc90\") " pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630017 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-cni-netd\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630059 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwtls\" (UniqueName: \"kubernetes.io/projected/31962401-01b7-4b65-aa86-e64db7ad9af4-kube-api-access-lwtls\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630085 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-cni-netd\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630131 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-cni-dir\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630245 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-run-k8s-cni-cncf-io\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630288 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-conf-dir\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630314 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-var-lib-kubelet\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630332 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630373 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhsdl\" (UniqueName: \"kubernetes.io/projected/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-kube-api-access-dhsdl\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630414 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-var-lib-cni-multus\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-kubernetes\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630467 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b77686d0-8cd0-4908-98fb-f34ff9a63c79-host\") pod \"node-ca-nxfwf\" (UID: \"b77686d0-8cd0-4908-98fb-f34ff9a63c79\") " pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630465 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-var-lib-kubelet\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.696309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630506 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630539 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-systemd-units\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630547 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-kubernetes\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630591 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-systemd-units\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630602 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b77686d0-8cd0-4908-98fb-f34ff9a63c79-host\") pod \"node-ca-nxfwf\" (UID: \"b77686d0-8cd0-4908-98fb-f34ff9a63c79\") " pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630621 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/31962401-01b7-4b65-aa86-e64db7ad9af4-ovn-node-metrics-cert\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630663 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-socket-dir-parent\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.630843 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-daemon-config\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631269 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631238 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmkwh\" (UniqueName: \"kubernetes.io/projected/d1de09de-782e-4b78-a68d-cbf1078cbc90-kube-api-access-pmkwh\") pod \"node-resolver-7tdk9\" (UID: \"d1de09de-782e-4b78-a68d-cbf1078cbc90\") " pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631746 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-run-netns\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631772 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-etc-openvswitch\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631775 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631801 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-device-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631822 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-etc-selinux\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631871 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-cnibin\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.700219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631893 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-run-multus-certs\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-run\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631940 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/31962401-01b7-4b65-aa86-e64db7ad9af4-ovnkube-config\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631961 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.631983 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-registration-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-sys-fs\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632050 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7832f593-a763-4b0f-be98-f5c73bec2465-iptables-alerter-script\") pod \"iptables-alerter-x9j2s\" (UID: \"7832f593-a763-4b0f-be98-f5c73bec2465\") " pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632058 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-run-netns\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632075 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps9tq\" (UniqueName: \"kubernetes.io/projected/fab8b243-8bf0-41c8-b56c-d7f75510b714-kube-api-access-ps9tq\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632102 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-cnibin\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632117 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-etc-openvswitch\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632127 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/94fbec68-3b05-4289-89d9-21b2a420a758-agent-certs\") pod \"konnectivity-agent-t6d5r\" (UID: \"94fbec68-3b05-4289-89d9-21b2a420a758\") " pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632152 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b77686d0-8cd0-4908-98fb-f34ff9a63c79-serviceca\") pod \"node-ca-nxfwf\" (UID: \"b77686d0-8cd0-4908-98fb-f34ff9a63c79\") " pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632211 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nb2g\" (UniqueName: \"kubernetes.io/projected/b77686d0-8cd0-4908-98fb-f34ff9a63c79-kube-api-access-8nb2g\") pod \"node-ca-nxfwf\" (UID: \"b77686d0-8cd0-4908-98fb-f34ff9a63c79\") " pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632236 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632263 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-var-lib-cni-bin\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632303 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-lib-modules\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.701094 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-host\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632337 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-tuned\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632371 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-tmp\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632396 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-run-openvswitch\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632422 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-node-log\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632462 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg4tc\" (UniqueName: \"kubernetes.io/projected/8c59e863-1727-4259-bda5-36fda7c6e44f-kube-api-access-gg4tc\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632487 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5fbl\" (UniqueName: \"kubernetes.io/projected/eaf250c9-d810-4a19-9844-b0e5fa5b8526-kube-api-access-t5fbl\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632516 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-systemd\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-os-release\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632640 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-run-systemd\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-var-lib-openvswitch\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632719 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-var-lib-kubelet\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632749 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-cni-bin\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632775 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-slash\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632803 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-log-socket\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-dbus\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632862 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-modprobe-d\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.701993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632897 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-run-ovn\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632921 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-run-netns\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632939 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/31962401-01b7-4b65-aa86-e64db7ad9af4-ovnkube-config\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632949 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-etc-kubernetes\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632976 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-sysctl-d\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632991 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633006 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1de09de-782e-4b78-a68d-cbf1078cbc90-hosts-file\") pod \"node-resolver-7tdk9\" (UID: \"d1de09de-782e-4b78-a68d-cbf1078cbc90\") " pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633033 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.632301 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-run\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633096 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-lib-modules\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633164 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-cnibin\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633206 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-slash\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqh8\" (UniqueName: \"kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8\") pod \"network-check-target-jfkr8\" (UID: \"e1f24e3f-ab56-4fa0-951b-45c19ed83baf\") " pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633271 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633296 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-hostroot\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633323 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/31962401-01b7-4b65-aa86-e64db7ad9af4-env-overrides\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633352 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/31962401-01b7-4b65-aa86-e64db7ad9af4-ovnkube-script-lib\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.703615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633371 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-log-socket\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633379 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fab8b243-8bf0-41c8-b56c-d7f75510b714-cni-binary-copy\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633409 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-node-log\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633408 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-sysctl-conf\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633470 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b77686d0-8cd0-4908-98fb-f34ff9a63c79-serviceca\") pod \"node-ca-nxfwf\" (UID: \"b77686d0-8cd0-4908-98fb-f34ff9a63c79\") " pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633517 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-run-openvswitch\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633518 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-run-ovn-kubernetes\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633600 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-sysctl-conf\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633689 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633724 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-socket-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633733 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-modprobe-d\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633753 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7832f593-a763-4b0f-be98-f5c73bec2465-host-slash\") pod \"iptables-alerter-x9j2s\" (UID: \"7832f593-a763-4b0f-be98-f5c73bec2465\") " pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633780 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633780 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/94fbec68-3b05-4289-89d9-21b2a420a758-konnectivity-ca\") pod \"konnectivity-agent-t6d5r\" (UID: \"94fbec68-3b05-4289-89d9-21b2a420a758\") " pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633832 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-run-ovn\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633868 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-run-ovn-kubernetes\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.633963 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-dbus\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:15.706720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.634010 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-sysctl-d\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.634018 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-host-cni-bin\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.634063 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d1de09de-782e-4b78-a68d-cbf1078cbc90-hosts-file\") pod \"node-resolver-7tdk9\" (UID: \"d1de09de-782e-4b78-a68d-cbf1078cbc90\") " pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.634112 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-systemd\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.634171 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-os-release\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.634211 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-host\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.634234 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-var-lib-openvswitch\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.634350 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/31962401-01b7-4b65-aa86-e64db7ad9af4-run-systemd\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.634586 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.635570 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/31962401-01b7-4b65-aa86-e64db7ad9af4-env-overrides\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.635754 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/31962401-01b7-4b65-aa86-e64db7ad9af4-ovnkube-script-lib\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.637166 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" event={"ID":"a836bcb889abc73302d1336d89ed5761","Type":"ContainerStarted","Data":"284d8d938bbef5e3352a950a131966b11415d32804f27964b54f9ddd5bf6e7ca"} Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.638224 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-etc-tuned\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.638358 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/31962401-01b7-4b65-aa86-e64db7ad9af4-ovn-node-metrics-cert\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.639171 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" event={"ID":"fb8eed94bb9bf5d97c7ef83964eedb6a","Type":"ContainerStarted","Data":"d3c51361daf468acbb4015059aa7e93cae97836f5e6472d48384f9bbf408b88e"} Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.640360 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr959\" (UniqueName: \"kubernetes.io/projected/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-kube-api-access-kr959\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.642790 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhsdl\" (UniqueName: \"kubernetes.io/projected/e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f-kube-api-access-dhsdl\") pod \"multus-additional-cni-plugins-8xzv4\" (UID: \"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f\") " pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.643248 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nb2g\" (UniqueName: \"kubernetes.io/projected/b77686d0-8cd0-4908-98fb-f34ff9a63c79-kube-api-access-8nb2g\") pod \"node-ca-nxfwf\" (UID: \"b77686d0-8cd0-4908-98fb-f34ff9a63c79\") " pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.707386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.643600 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb85ef6a-72e5-48b9-854c-e2074fbbfdea-tmp\") pod \"tuned-j2c2g\" (UID: \"fb85ef6a-72e5-48b9-854c-e2074fbbfdea\") " pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.709168 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.644244 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:40:15.709168 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.644269 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:40:15.709168 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.644294 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hnqh8 for pod openshift-network-diagnostics/network-check-target-jfkr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:15.709168 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.644364 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8 podName:e1f24e3f-ab56-4fa0-951b-45c19ed83baf nodeName:}" failed. No retries permitted until 2026-07-22 16:40:16.144345198 +0000 UTC m=+3.055002354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hnqh8" (UniqueName: "kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8") pod "network-check-target-jfkr8" (UID: "e1f24e3f-ab56-4fa0-951b-45c19ed83baf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:15.709168 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.648336 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmkwh\" (UniqueName: \"kubernetes.io/projected/d1de09de-782e-4b78-a68d-cbf1078cbc90-kube-api-access-pmkwh\") pod \"node-resolver-7tdk9\" (UID: \"d1de09de-782e-4b78-a68d-cbf1078cbc90\") " pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.709168 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.655303 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwtls\" (UniqueName: \"kubernetes.io/projected/31962401-01b7-4b65-aa86-e64db7ad9af4-kube-api-access-lwtls\") pod \"ovnkube-node-dsxzr\" (UID: \"31962401-01b7-4b65-aa86-e64db7ad9af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.734648 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-run-netns\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.734681 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-etc-kubernetes\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.734724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.734792 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-etc-kubernetes\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.734834 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.734866 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-run-netns\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.734924 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-hostroot\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.734956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fab8b243-8bf0-41c8-b56c-d7f75510b714-cni-binary-copy\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.734984 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-socket-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735008 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7832f593-a763-4b0f-be98-f5c73bec2465-host-slash\") pod \"iptables-alerter-x9j2s\" (UID: \"7832f593-a763-4b0f-be98-f5c73bec2465\") " pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:15.735053 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs podName:eaf250c9-d810-4a19-9844-b0e5fa5b8526 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:16.235032486 +0000 UTC m=+3.145689644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs") pod "network-metrics-daemon-ld4x4" (UID: "eaf250c9-d810-4a19-9844-b0e5fa5b8526") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735075 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/94fbec68-3b05-4289-89d9-21b2a420a758-konnectivity-ca\") pod \"konnectivity-agent-t6d5r\" (UID: \"94fbec68-3b05-4289-89d9-21b2a420a758\") " pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735075 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-hostroot\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735133 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n27tg\" (UniqueName: \"kubernetes.io/projected/7832f593-a763-4b0f-be98-f5c73bec2465-kube-api-access-n27tg\") pod \"iptables-alerter-x9j2s\" (UID: \"7832f593-a763-4b0f-be98-f5c73bec2465\") " pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735147 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-socket-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735161 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-system-cni-dir\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735163 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7832f593-a763-4b0f-be98-f5c73bec2465-host-slash\") pod \"iptables-alerter-x9j2s\" (UID: \"7832f593-a763-4b0f-be98-f5c73bec2465\") " pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.797982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735187 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-os-release\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735217 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-cni-dir\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735239 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-run-k8s-cni-cncf-io\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735264 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-conf-dir\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735289 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-var-lib-cni-multus\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-socket-dir-parent\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735340 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-daemon-config\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-device-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735414 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-etc-selinux\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735438 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-cnibin\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735461 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-run-multus-certs\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735485 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-system-cni-dir\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735487 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735532 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-registration-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735542 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735559 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-sys-fs\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735586 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-conf-dir\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735586 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7832f593-a763-4b0f-be98-f5c73bec2465-iptables-alerter-script\") pod \"iptables-alerter-x9j2s\" (UID: \"7832f593-a763-4b0f-be98-f5c73bec2465\") " pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735627 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps9tq\" (UniqueName: \"kubernetes.io/projected/fab8b243-8bf0-41c8-b56c-d7f75510b714-kube-api-access-ps9tq\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/94fbec68-3b05-4289-89d9-21b2a420a758-agent-certs\") pod \"konnectivity-agent-t6d5r\" (UID: \"94fbec68-3b05-4289-89d9-21b2a420a758\") " pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735685 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-var-lib-cni-bin\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735718 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg4tc\" (UniqueName: \"kubernetes.io/projected/8c59e863-1727-4259-bda5-36fda7c6e44f-kube-api-access-gg4tc\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735719 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/94fbec68-3b05-4289-89d9-21b2a420a758-konnectivity-ca\") pod \"konnectivity-agent-t6d5r\" (UID: \"94fbec68-3b05-4289-89d9-21b2a420a758\") " pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735437 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-run-k8s-cni-cncf-io\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735771 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-cnibin\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735789 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-run-multus-certs\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735789 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fab8b243-8bf0-41c8-b56c-d7f75510b714-cni-binary-copy\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735842 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-var-lib-cni-multus\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735874 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-etc-selinux\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735871 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-socket-dir-parent\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735906 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5fbl\" (UniqueName: \"kubernetes.io/projected/eaf250c9-d810-4a19-9844-b0e5fa5b8526-kube-api-access-t5fbl\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735921 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-cni-dir\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735923 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-registration-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735942 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-var-lib-kubelet\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735956 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-sys-fs\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.805933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735980 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-var-lib-cni-bin\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.806613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.735989 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-os-release\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.806613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.736011 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fab8b243-8bf0-41c8-b56c-d7f75510b714-host-var-lib-kubelet\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.806613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.736029 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c59e863-1727-4259-bda5-36fda7c6e44f-device-dir\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.806613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.736068 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7832f593-a763-4b0f-be98-f5c73bec2465-iptables-alerter-script\") pod \"iptables-alerter-x9j2s\" (UID: \"7832f593-a763-4b0f-be98-f5c73bec2465\") " pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.806613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.736524 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fab8b243-8bf0-41c8-b56c-d7f75510b714-multus-daemon-config\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.806613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.738438 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/94fbec68-3b05-4289-89d9-21b2a420a758-agent-certs\") pod \"konnectivity-agent-t6d5r\" (UID: \"94fbec68-3b05-4289-89d9-21b2a420a758\") " pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:15.806613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.755059 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5fbl\" (UniqueName: \"kubernetes.io/projected/eaf250c9-d810-4a19-9844-b0e5fa5b8526-kube-api-access-t5fbl\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:15.806613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.756926 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps9tq\" (UniqueName: \"kubernetes.io/projected/fab8b243-8bf0-41c8-b56c-d7f75510b714-kube-api-access-ps9tq\") pod \"multus-qj8th\" (UID: \"fab8b243-8bf0-41c8-b56c-d7f75510b714\") " pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.806613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.757273 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n27tg\" (UniqueName: \"kubernetes.io/projected/7832f593-a763-4b0f-be98-f5c73bec2465-kube-api-access-n27tg\") pod \"iptables-alerter-x9j2s\" (UID: \"7832f593-a763-4b0f-be98-f5c73bec2465\") " pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.806613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.758558 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg4tc\" (UniqueName: \"kubernetes.io/projected/8c59e863-1727-4259-bda5-36fda7c6e44f-kube-api-access-gg4tc\") pod \"aws-ebs-csi-driver-node-tpcr6\" (UID: \"8c59e863-1727-4259-bda5-36fda7c6e44f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:15.888426 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.817260 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" Jul 22 16:40:15.888426 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:15.830913 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb85ef6a_72e5_48b9_854c_e2074fbbfdea.slice/crio-0a09b4ae52e1a862e532072f520e419820f906a13a8b0873780f0913ba9e16e2 WatchSource:0}: Error finding container 0a09b4ae52e1a862e532072f520e419820f906a13a8b0873780f0913ba9e16e2: Status 404 returned error can't find the container with id 0a09b4ae52e1a862e532072f520e419820f906a13a8b0873780f0913ba9e16e2 Jul 22 16:40:15.888426 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.833138 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nxfwf" Jul 22 16:40:15.888426 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.842177 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:40:15.888426 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.844005 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7tdk9" Jul 22 16:40:15.888426 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:15.844325 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb77686d0_8cd0_4908_98fb_f34ff9a63c79.slice/crio-18d472c274f6c82a0a556af584e51d7224b660fb740aa706f5e9d6eb771eba97 WatchSource:0}: Error finding container 18d472c274f6c82a0a556af584e51d7224b660fb740aa706f5e9d6eb771eba97: Status 404 returned error can't find the container with id 18d472c274f6c82a0a556af584e51d7224b660fb740aa706f5e9d6eb771eba97 Jul 22 16:40:15.888426 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.851693 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" Jul 22 16:40:15.888426 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.860082 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:15.888426 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.869702 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qj8th" Jul 22 16:40:15.902412 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:15.902385 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1de09de_782e_4b78_a68d_cbf1078cbc90.slice/crio-2972c8327f565ebf76a710602253c75b3099bfe24fd16793bb19bc96ba371420 WatchSource:0}: Error finding container 2972c8327f565ebf76a710602253c75b3099bfe24fd16793bb19bc96ba371420: Status 404 returned error can't find the container with id 2972c8327f565ebf76a710602253c75b3099bfe24fd16793bb19bc96ba371420 Jul 22 16:40:15.904132 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:15.904100 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b3ef7b_fe5d_41c9_9e86_9f5287a5496f.slice/crio-b37468a7f9a3ddfb567113c0769e72c791260e27555dbb761fbf66ceb27031dc WatchSource:0}: Error finding container b37468a7f9a3ddfb567113c0769e72c791260e27555dbb761fbf66ceb27031dc: Status 404 returned error can't find the container with id b37468a7f9a3ddfb567113c0769e72c791260e27555dbb761fbf66ceb27031dc Jul 22 16:40:15.909504 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.909311 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x9j2s" Jul 22 16:40:15.918687 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.918666 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:15.919435 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:15.919276 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab8b243_8bf0_41c8_b56c_d7f75510b714.slice/crio-663e2e5fa39532f83852c7b4eca8f8ee56bb914b007162cfe0b79189b9582bd8 WatchSource:0}: Error finding container 663e2e5fa39532f83852c7b4eca8f8ee56bb914b007162cfe0b79189b9582bd8: Status 404 returned error can't find the container with id 663e2e5fa39532f83852c7b4eca8f8ee56bb914b007162cfe0b79189b9582bd8 Jul 22 16:40:15.919785 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:15.919759 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31962401_01b7_4b65_aa86_e64db7ad9af4.slice/crio-75b0714037f43ffc2b4d1280fbe004c867ab46019d6a2395422c2d8726c5b70f WatchSource:0}: Error finding container 75b0714037f43ffc2b4d1280fbe004c867ab46019d6a2395422c2d8726c5b70f: Status 404 returned error can't find the container with id 75b0714037f43ffc2b4d1280fbe004c867ab46019d6a2395422c2d8726c5b70f Jul 22 16:40:15.928338 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:15.928314 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7832f593_a763_4b0f_be98_f5c73bec2465.slice/crio-4471c996285886ce385a09b51cc5cabf37762786108bace549af7d65c2d068d0 WatchSource:0}: Error finding container 4471c996285886ce385a09b51cc5cabf37762786108bace549af7d65c2d068d0: Status 404 returned error can't find the container with id 4471c996285886ce385a09b51cc5cabf37762786108bace549af7d65c2d068d0 Jul 22 16:40:15.936320 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:15.936294 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" Jul 22 16:40:16.139539 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.139445 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:16.139693 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:16.139591 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:16.139693 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:16.139659 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret podName:e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a nodeName:}" failed. No retries permitted until 2026-07-22 16:40:17.13963987 +0000 UTC m=+4.050297041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret") pod "global-pull-secret-syncer-dmtxl" (UID: "e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:16.240220 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.240185 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqh8\" (UniqueName: \"kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8\") pod \"network-check-target-jfkr8\" (UID: \"e1f24e3f-ab56-4fa0-951b-45c19ed83baf\") " pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:16.240412 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.240230 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:16.240479 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:16.240412 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:40:16.240479 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:16.240438 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:40:16.240479 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:16.240454 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hnqh8 for pod openshift-network-diagnostics/network-check-target-jfkr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:16.240618 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:16.240483 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:16.240618 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:16.240523 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8 podName:e1f24e3f-ab56-4fa0-951b-45c19ed83baf nodeName:}" failed. No retries permitted until 2026-07-22 16:40:17.240501534 +0000 UTC m=+4.151158689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hnqh8" (UniqueName: "kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8") pod "network-check-target-jfkr8" (UID: "e1f24e3f-ab56-4fa0-951b-45c19ed83baf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:16.240618 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:16.240546 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs podName:eaf250c9-d810-4a19-9844-b0e5fa5b8526 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:17.240533701 +0000 UTC m=+4.151190857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs") pod "network-metrics-daemon-ld4x4" (UID: "eaf250c9-d810-4a19-9844-b0e5fa5b8526") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:16.561017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.560932 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 16:35:16 +0000 UTC" deadline="2028-02-03 03:17:02.824301765 +0000 UTC" Jul 22 16:40:16.561017 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.560968 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13450h36m46.263338206s" Jul 22 16:40:16.612591 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.612559 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:16.612737 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:16.612688 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:16.642794 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.642761 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7tdk9" event={"ID":"d1de09de-782e-4b78-a68d-cbf1078cbc90","Type":"ContainerStarted","Data":"2972c8327f565ebf76a710602253c75b3099bfe24fd16793bb19bc96ba371420"} Jul 22 16:40:16.643994 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.643958 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nxfwf" event={"ID":"b77686d0-8cd0-4908-98fb-f34ff9a63c79","Type":"ContainerStarted","Data":"18d472c274f6c82a0a556af584e51d7224b660fb740aa706f5e9d6eb771eba97"} Jul 22 16:40:16.645380 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.645339 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" event={"ID":"fb85ef6a-72e5-48b9-854c-e2074fbbfdea","Type":"ContainerStarted","Data":"0a09b4ae52e1a862e532072f520e419820f906a13a8b0873780f0913ba9e16e2"} Jul 22 16:40:16.646723 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.646697 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x9j2s" event={"ID":"7832f593-a763-4b0f-be98-f5c73bec2465","Type":"ContainerStarted","Data":"4471c996285886ce385a09b51cc5cabf37762786108bace549af7d65c2d068d0"} Jul 22 16:40:16.647935 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.647908 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qj8th" event={"ID":"fab8b243-8bf0-41c8-b56c-d7f75510b714","Type":"ContainerStarted","Data":"663e2e5fa39532f83852c7b4eca8f8ee56bb914b007162cfe0b79189b9582bd8"} Jul 22 16:40:16.649242 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.649216 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" event={"ID":"31962401-01b7-4b65-aa86-e64db7ad9af4","Type":"ContainerStarted","Data":"75b0714037f43ffc2b4d1280fbe004c867ab46019d6a2395422c2d8726c5b70f"} Jul 22 16:40:16.650570 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:16.650541 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" event={"ID":"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f","Type":"ContainerStarted","Data":"b37468a7f9a3ddfb567113c0769e72c791260e27555dbb761fbf66ceb27031dc"} Jul 22 16:40:16.705660 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:16.705614 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94fbec68_3b05_4289_89d9_21b2a420a758.slice/crio-61958d4c31eea1ee0bed0915d1cf72022486e34acd51897c56b785d5764fe2c9 WatchSource:0}: Error finding container 61958d4c31eea1ee0bed0915d1cf72022486e34acd51897c56b785d5764fe2c9: Status 404 returned error can't find the container with id 61958d4c31eea1ee0bed0915d1cf72022486e34acd51897c56b785d5764fe2c9 Jul 22 16:40:17.146365 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.146323 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:17.146580 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:17.146525 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:17.146658 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:17.146589 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret podName:e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a nodeName:}" failed. No retries permitted until 2026-07-22 16:40:19.146569135 +0000 UTC m=+6.057226293 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret") pod "global-pull-secret-syncer-dmtxl" (UID: "e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:17.250491 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.246731 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqh8\" (UniqueName: \"kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8\") pod \"network-check-target-jfkr8\" (UID: \"e1f24e3f-ab56-4fa0-951b-45c19ed83baf\") " pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:17.250491 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.246784 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:17.250491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:17.246946 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:17.250491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:17.247007 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:40:17.250491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:17.247040 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:40:17.250491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:17.247057 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hnqh8 for pod openshift-network-diagnostics/network-check-target-jfkr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:17.250491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:17.247018 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs podName:eaf250c9-d810-4a19-9844-b0e5fa5b8526 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:19.246997691 +0000 UTC m=+6.157654848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs") pod "network-metrics-daemon-ld4x4" (UID: "eaf250c9-d810-4a19-9844-b0e5fa5b8526") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:17.250491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:17.247188 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8 podName:e1f24e3f-ab56-4fa0-951b-45c19ed83baf nodeName:}" failed. No retries permitted until 2026-07-22 16:40:19.247163328 +0000 UTC m=+6.157820485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hnqh8" (UniqueName: "kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8") pod "network-check-target-jfkr8" (UID: "e1f24e3f-ab56-4fa0-951b-45c19ed83baf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:17.293202 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:17.293173 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c59e863_1727_4259_bda5_36fda7c6e44f.slice/crio-8a844674bc2d0535af9af17dea97aefbfe44d54140a557e23e9d8acb8dfd6ace WatchSource:0}: Error finding container 8a844674bc2d0535af9af17dea97aefbfe44d54140a557e23e9d8acb8dfd6ace: Status 404 returned error can't find the container with id 8a844674bc2d0535af9af17dea97aefbfe44d54140a557e23e9d8acb8dfd6ace Jul 22 16:40:17.561515 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.561397 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 16:35:16 +0000 UTC" deadline="2028-05-01 11:26:52.046879898 +0000 UTC" Jul 22 16:40:17.561515 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.561514 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15570h46m34.485371198s" Jul 22 16:40:17.617147 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.617116 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:17.617311 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:17.617251 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:17.617849 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.617826 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:17.617976 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:17.617954 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:17.661088 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.660790 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" event={"ID":"8c59e863-1727-4259-bda5-36fda7c6e44f","Type":"ContainerStarted","Data":"8a844674bc2d0535af9af17dea97aefbfe44d54140a557e23e9d8acb8dfd6ace"} Jul 22 16:40:17.664021 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.663989 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t6d5r" event={"ID":"94fbec68-3b05-4289-89d9-21b2a420a758","Type":"ContainerStarted","Data":"61958d4c31eea1ee0bed0915d1cf72022486e34acd51897c56b785d5764fe2c9"} Jul 22 16:40:17.671783 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.671753 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" event={"ID":"fb8eed94bb9bf5d97c7ef83964eedb6a","Type":"ContainerStarted","Data":"3201504d5883ea76b93962cea801c0892ead554ab3489b34ad546123cb10b0b6"} Jul 22 16:40:17.696222 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:17.696166 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" podStartSLOduration=2.696149351 podStartE2EDuration="2.696149351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:17.695500801 +0000 UTC m=+4.606157977" watchObservedRunningTime="2026-07-22 16:40:17.696149351 +0000 UTC m=+4.606806532" Jul 22 16:40:18.613058 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:18.613018 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:18.613987 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:18.613197 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:18.692015 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:18.691759 2581 generic.go:356] "Generic (PLEG): container finished" podID="a836bcb889abc73302d1336d89ed5761" containerID="69dc8aa6ed6add1ccd394d0f6a7886892d868c2a2817135c4fc08cb2da4557db" exitCode=0 Jul 22 16:40:18.693020 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:18.692744 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" event={"ID":"a836bcb889abc73302d1336d89ed5761","Type":"ContainerDied","Data":"69dc8aa6ed6add1ccd394d0f6a7886892d868c2a2817135c4fc08cb2da4557db"} Jul 22 16:40:19.163592 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:19.163512 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:19.163760 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:19.163631 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:19.163760 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:19.163682 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret podName:e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a nodeName:}" failed. No retries permitted until 2026-07-22 16:40:23.163668961 +0000 UTC m=+10.074326113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret") pod "global-pull-secret-syncer-dmtxl" (UID: "e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:19.264946 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:19.264068 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqh8\" (UniqueName: \"kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8\") pod \"network-check-target-jfkr8\" (UID: \"e1f24e3f-ab56-4fa0-951b-45c19ed83baf\") " pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:19.264946 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:19.264120 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:19.264946 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:19.264310 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:19.264946 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:19.264376 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs podName:eaf250c9-d810-4a19-9844-b0e5fa5b8526 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:23.264355312 +0000 UTC m=+10.175012471 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs") pod "network-metrics-daemon-ld4x4" (UID: "eaf250c9-d810-4a19-9844-b0e5fa5b8526") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:19.264946 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:19.264827 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:40:19.264946 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:19.264849 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:40:19.264946 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:19.264862 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hnqh8 for pod openshift-network-diagnostics/network-check-target-jfkr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:19.264946 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:19.264909 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8 podName:e1f24e3f-ab56-4fa0-951b-45c19ed83baf nodeName:}" failed. No retries permitted until 2026-07-22 16:40:23.264892224 +0000 UTC m=+10.175549382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hnqh8" (UniqueName: "kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8") pod "network-check-target-jfkr8" (UID: "e1f24e3f-ab56-4fa0-951b-45c19ed83baf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:19.629818 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:19.629493 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:19.629818 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:19.629715 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:19.630773 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:19.630539 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:19.630773 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:19.630716 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:19.698548 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:19.698463 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" event={"ID":"a836bcb889abc73302d1336d89ed5761","Type":"ContainerStarted","Data":"6a27a76f51616af1fc8e5a054cda23c84026385b2cfedd052b768d9c9dc9f975"} Jul 22 16:40:20.612727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:20.612694 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:20.612921 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:20.612837 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:21.612772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:21.612671 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:21.612772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:21.612707 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:21.613294 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:21.612829 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:21.613294 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:21.613057 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:22.612515 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:22.612471 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:22.612704 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:22.612609 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:23.199652 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:23.199610 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:23.200144 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:23.199793 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:23.200144 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:23.199862 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret podName:e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a nodeName:}" failed. No retries permitted until 2026-07-22 16:40:31.199841405 +0000 UTC m=+18.110498575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret") pod "global-pull-secret-syncer-dmtxl" (UID: "e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:23.300812 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:23.300445 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqh8\" (UniqueName: \"kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8\") pod \"network-check-target-jfkr8\" (UID: \"e1f24e3f-ab56-4fa0-951b-45c19ed83baf\") " pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:23.300812 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:23.300511 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:23.300812 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:23.300612 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:40:23.300812 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:23.300646 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:40:23.300812 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:23.300661 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hnqh8 for pod openshift-network-diagnostics/network-check-target-jfkr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:23.300812 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:23.300709 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:23.300812 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:23.300727 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8 podName:e1f24e3f-ab56-4fa0-951b-45c19ed83baf nodeName:}" failed. No retries permitted until 2026-07-22 16:40:31.300706768 +0000 UTC m=+18.211363936 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hnqh8" (UniqueName: "kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8") pod "network-check-target-jfkr8" (UID: "e1f24e3f-ab56-4fa0-951b-45c19ed83baf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:23.300812 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:23.300777 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs podName:eaf250c9-d810-4a19-9844-b0e5fa5b8526 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:31.300753876 +0000 UTC m=+18.211411033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs") pod "network-metrics-daemon-ld4x4" (UID: "eaf250c9-d810-4a19-9844-b0e5fa5b8526") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:23.614072 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:23.613981 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:23.614072 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:23.614009 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:23.614288 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:23.614113 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:23.614288 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:23.614257 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:24.613022 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:24.612978 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:24.613579 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:24.613122 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:25.613098 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:25.613012 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:25.613593 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:25.613194 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:25.613662 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:25.613622 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:25.613759 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:25.613736 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:26.613202 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:26.613163 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:26.613668 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:26.613290 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:27.613056 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:27.613021 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:27.613234 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:27.613144 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:27.613234 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:27.613193 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:27.613659 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:27.613290 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:28.612807 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:28.612765 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:28.613001 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:28.612898 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:29.613167 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:29.613134 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:29.614004 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:29.613270 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:29.614004 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:29.613328 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:29.614004 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:29.613458 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:30.613261 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:30.613224 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:30.613795 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:30.613346 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:31.267984 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:31.267944 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:31.268259 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:31.268078 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:31.268259 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:31.268146 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret podName:e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a nodeName:}" failed. No retries permitted until 2026-07-22 16:40:47.26812562 +0000 UTC m=+34.178782811 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret") pod "global-pull-secret-syncer-dmtxl" (UID: "e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:31.369052 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:31.369012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqh8\" (UniqueName: \"kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8\") pod \"network-check-target-jfkr8\" (UID: \"e1f24e3f-ab56-4fa0-951b-45c19ed83baf\") " pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:31.369052 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:31.369055 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:31.369285 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:31.369188 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:31.369285 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:31.369240 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs podName:eaf250c9-d810-4a19-9844-b0e5fa5b8526 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:47.369226625 +0000 UTC m=+34.279883774 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs") pod "network-metrics-daemon-ld4x4" (UID: "eaf250c9-d810-4a19-9844-b0e5fa5b8526") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:31.369285 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:31.369191 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:40:31.369285 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:31.369283 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:40:31.369510 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:31.369297 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hnqh8 for pod openshift-network-diagnostics/network-check-target-jfkr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:31.369510 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:31.369341 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8 podName:e1f24e3f-ab56-4fa0-951b-45c19ed83baf nodeName:}" failed. No retries permitted until 2026-07-22 16:40:47.369329187 +0000 UTC m=+34.279986354 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hnqh8" (UniqueName: "kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8") pod "network-check-target-jfkr8" (UID: "e1f24e3f-ab56-4fa0-951b-45c19ed83baf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:31.612640 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:31.612559 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:31.612793 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:31.612562 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:31.612793 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:31.612681 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:31.612879 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:31.612797 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:32.613444 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:32.613413 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:32.613870 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:32.613531 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:33.613917 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:33.613884 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:33.614452 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:33.613992 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:33.614452 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:33.614052 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:33.614452 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:33.614151 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:34.612520 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:34.612490 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:34.612679 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:34.612597 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:35.612838 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:35.612807 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:35.613190 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:35.612909 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:35.613190 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:35.613001 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:35.613190 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:35.613123 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:36.613479 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.613199 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:36.614005 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:36.613552 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:36.729012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.728987 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7tdk9" event={"ID":"d1de09de-782e-4b78-a68d-cbf1078cbc90","Type":"ContainerStarted","Data":"a69003b7b9462bad1fe72053232b96b53721e573e100b016070059e5675e4fd8"} Jul 22 16:40:36.730282 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.730259 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nxfwf" event={"ID":"b77686d0-8cd0-4908-98fb-f34ff9a63c79","Type":"ContainerStarted","Data":"87f9e18de2fd0bdee07f35438ad8439051130b8fb73922428885f6e83758101c"} Jul 22 16:40:36.731600 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.731581 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" event={"ID":"fb85ef6a-72e5-48b9-854c-e2074fbbfdea","Type":"ContainerStarted","Data":"c53908be098f999a281a7294b83990c3716651b9a7d7ac8878119a39b4d4cce6"} Jul 22 16:40:36.732724 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.732705 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qj8th" event={"ID":"fab8b243-8bf0-41c8-b56c-d7f75510b714","Type":"ContainerStarted","Data":"6b4fd090654e4b44dad958cd94af8129c2cfb8198b42dd17bb077fe4b77d2910"} Jul 22 16:40:36.735027 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.735003 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" event={"ID":"31962401-01b7-4b65-aa86-e64db7ad9af4","Type":"ContainerStarted","Data":"c085c3d0c0ed92c0f0a3a3f100504cb3d4b948e5e551be4229e47a1bd230ff47"} Jul 22 16:40:36.735108 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.735056 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" event={"ID":"31962401-01b7-4b65-aa86-e64db7ad9af4","Type":"ContainerStarted","Data":"9a226ef391fdfd9c20fae002f517db4aeca14e66c4ae2f68a7a5147c3fea3db1"} Jul 22 16:40:36.735108 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.735100 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" event={"ID":"31962401-01b7-4b65-aa86-e64db7ad9af4","Type":"ContainerStarted","Data":"9143c700c008099e625a31512b5b4f55e3ee1aac830e4c5187b251f9f9d625e3"} Jul 22 16:40:36.735192 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.735139 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" event={"ID":"31962401-01b7-4b65-aa86-e64db7ad9af4","Type":"ContainerStarted","Data":"3eb4088c8c2975f34334f2902e12604e13525803924505cd9200e9fd600edb68"} Jul 22 16:40:36.735192 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.735170 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" event={"ID":"31962401-01b7-4b65-aa86-e64db7ad9af4","Type":"ContainerStarted","Data":"80bd3644ba950b57ae41d045304872e6ef1095e11af9980201e7fb5dcdc9d24f"} Jul 22 16:40:36.736386 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.736349 2581 generic.go:356] "Generic (PLEG): container finished" podID="e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f" containerID="f231ce73f5f014da2b2256d266b1831233d8264792ab91735ea46b77e7477533" exitCode=0 Jul 22 16:40:36.736475 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.736393 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" event={"ID":"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f","Type":"ContainerDied","Data":"f231ce73f5f014da2b2256d266b1831233d8264792ab91735ea46b77e7477533"} Jul 22 16:40:36.738287 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.738264 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" event={"ID":"8c59e863-1727-4259-bda5-36fda7c6e44f","Type":"ContainerStarted","Data":"9d1d6d641e1b85a53bdccd6c143e3ca696e4f81e3fd86eb08909e3e9fdd4d1c2"} Jul 22 16:40:36.739634 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.739616 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t6d5r" event={"ID":"94fbec68-3b05-4289-89d9-21b2a420a758","Type":"ContainerStarted","Data":"9db84a5fc851ebdcfd681dcd0879fcbddf6365e7244bf4cce806a5150e8bf770"} Jul 22 16:40:36.760007 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.759969 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" podStartSLOduration=21.759958062 podStartE2EDuration="21.759958062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:19.730269469 +0000 UTC m=+6.640926646" watchObservedRunningTime="2026-07-22 16:40:36.759958062 +0000 UTC m=+23.670615233" Jul 22 16:40:36.778502 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.778470 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nxfwf" podStartSLOduration=2.541016267 podStartE2EDuration="22.778458551s" totalImagesPullingTime="20.237442284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:36.777815239 +0000 UTC m=+23.688472410" watchObservedRunningTime="2026-07-22 16:40:36.778458551 +0000 UTC m=+23.689115724" Jul 22 16:40:36.793790 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.793756 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j2c2g" podStartSLOduration=2.547626501 podStartE2EDuration="22.793744611s" totalImagesPullingTime="20.24611811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:36.79365972 +0000 UTC m=+23.704316892" watchObservedRunningTime="2026-07-22 16:40:36.793744611 +0000 UTC m=+23.704401823" Jul 22 16:40:36.845069 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.844987 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7tdk9" podStartSLOduration=2.665737818 podStartE2EDuration="22.84496982s" totalImagesPullingTime="20.179232002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:36.810224681 +0000 UTC m=+23.720881854" watchObservedRunningTime="2026-07-22 16:40:36.84496982 +0000 UTC m=+23.755627022" Jul 22 16:40:36.845208 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:36.845161 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qj8th" podStartSLOduration=2.5575298650000002 podStartE2EDuration="22.845152177s" totalImagesPullingTime="20.287622312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:36.845123303 +0000 UTC m=+23.755780474" watchObservedRunningTime="2026-07-22 16:40:36.845152177 +0000 UTC m=+23.755809350" Jul 22 16:40:37.559460 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.559438 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 16:40:37.583662 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.583534 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T16:40:37.55945391Z","UUID":"025a223c-0b46-4e8e-ba7a-28ad16b2abe8","Handler":null,"Name":"","Endpoint":""} Jul 22 16:40:37.584736 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.584714 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 16:40:37.584816 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.584758 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 16:40:37.612855 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.612831 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:37.612971 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:37.612947 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:37.613049 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.613038 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:37.613156 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:37.613137 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:37.743385 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.743292 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x9j2s" event={"ID":"7832f593-a763-4b0f-be98-f5c73bec2465","Type":"ContainerStarted","Data":"94fbb7a89b9c9a48b6e89afc4699ee36ee36552997182024da30440c5ddc0bfb"} Jul 22 16:40:37.746872 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.746815 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" event={"ID":"31962401-01b7-4b65-aa86-e64db7ad9af4","Type":"ContainerStarted","Data":"82b42cded0875254eb985d6a4deaf08e4fdacab6c6ea1c48ce01dcbea6034c6f"} Jul 22 16:40:37.748836 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.748807 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" event={"ID":"8c59e863-1727-4259-bda5-36fda7c6e44f","Type":"ContainerStarted","Data":"3e0c532cd6f58bb52a939dccb29d443be59fd1a210394d43c919f44aa9b6c30e"} Jul 22 16:40:37.758937 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.758881 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t6d5r" podStartSLOduration=4.40041058 podStartE2EDuration="23.758863733s" totalImagesPullingTime="19.358453153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:36.859486152 +0000 UTC m=+23.770143324" watchObservedRunningTime="2026-07-22 16:40:37.758863733 +0000 UTC m=+24.669520903" Jul 22 16:40:37.759442 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:37.759410 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x9j2s" podStartSLOduration=3.609552469 podStartE2EDuration="23.759400167s" totalImagesPullingTime="20.149847698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:37.759060234 +0000 UTC m=+24.669717407" watchObservedRunningTime="2026-07-22 16:40:37.759400167 +0000 UTC m=+24.670057340" Jul 22 16:40:38.183593 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:38.183514 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:38.184099 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:38.184075 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:38.613447 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:38.613381 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:38.613670 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:38.613510 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:38.760890 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:38.760669 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:38.761352 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:38.761297 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t6d5r" Jul 22 16:40:39.612722 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:39.612688 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:39.612878 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:39.612803 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:39.612939 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:39.612876 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:39.613015 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:39.612993 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:39.756919 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:39.756876 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" event={"ID":"31962401-01b7-4b65-aa86-e64db7ad9af4","Type":"ContainerStarted","Data":"00200b9ea9ac289d4f04737374aac5f493450d2d6ab47f1f21ee0b5c45a4610f"} Jul 22 16:40:39.759224 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:39.759195 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" event={"ID":"8c59e863-1727-4259-bda5-36fda7c6e44f","Type":"ContainerStarted","Data":"29f46890f802b830c10e40a340b37f179790c869201ffc338e1c583c4b40f926"} Jul 22 16:40:39.779769 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:39.779726 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tpcr6" podStartSLOduration=4.762627168 podStartE2EDuration="25.779713233s" totalImagesPullingTime="21.017086065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:39.779266517 +0000 UTC m=+26.689923691" watchObservedRunningTime="2026-07-22 16:40:39.779713233 +0000 UTC m=+26.690370438" Jul 22 16:40:40.612486 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:40.612452 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:40.612649 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:40.612559 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:41.613648 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:41.613497 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:41.614235 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:41.613559 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:41.614235 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:41.613723 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:41.614235 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:41.613792 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:41.766132 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:41.766109 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" event={"ID":"31962401-01b7-4b65-aa86-e64db7ad9af4","Type":"ContainerStarted","Data":"c4504df9e1fc2673179f4c05ab667f8c6cc2bc46ee571432d2f161b15c934fc8"} Jul 22 16:40:41.766295 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:41.766275 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:41.766391 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:41.766352 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:41.767993 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:41.767972 2581 generic.go:356] "Generic (PLEG): container finished" podID="e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f" containerID="5175f3cb721efb05c6329a9af6df02defbccea395075b8efad32dbafb7b706ba" exitCode=0 Jul 22 16:40:41.768073 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:41.768014 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" event={"ID":"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f","Type":"ContainerDied","Data":"5175f3cb721efb05c6329a9af6df02defbccea395075b8efad32dbafb7b706ba"} Jul 22 16:40:41.781272 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:41.781252 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:41.799640 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:41.799566 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" podStartSLOduration=7.590525572 podStartE2EDuration="27.799553811s" totalImagesPullingTime="20.209028239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:41.79830358 +0000 UTC m=+28.708960763" watchObservedRunningTime="2026-07-22 16:40:41.799553811 +0000 UTC m=+28.710210998" Jul 22 16:40:42.612724 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:42.612693 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:42.612949 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:42.612805 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:42.770753 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:42.770724 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:42.784740 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:42.784717 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:40:43.193984 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:43.193954 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ld4x4"] Jul 22 16:40:43.194130 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:43.194064 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:43.194193 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:43.194169 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:43.198023 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:43.197997 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dmtxl"] Jul 22 16:40:43.198141 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:43.198100 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:43.198233 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:43.198191 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:43.198587 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:43.198567 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jfkr8"] Jul 22 16:40:43.198689 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:43.198676 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:43.198776 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:43.198758 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:43.774137 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:43.774107 2581 generic.go:356] "Generic (PLEG): container finished" podID="e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f" containerID="7ed8d6bbecbfed841125611c8e70369286ce6bfaff4d6c2623a8a7ee869905c0" exitCode=0 Jul 22 16:40:43.774603 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:43.774200 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" event={"ID":"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f","Type":"ContainerDied","Data":"7ed8d6bbecbfed841125611c8e70369286ce6bfaff4d6c2623a8a7ee869905c0"} Jul 22 16:40:44.613227 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:44.613200 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:44.613227 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:44.613200 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:44.613506 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:44.613306 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:44.613506 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:44.613448 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:45.613521 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:45.613456 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:45.613845 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:45.613562 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:45.781240 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:45.781198 2581 generic.go:356] "Generic (PLEG): container finished" podID="e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f" containerID="9a50c6347d164eae2e9a1158e12e66f2176d14e59a73438431f11acdd59d3a36" exitCode=0 Jul 22 16:40:45.781413 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:45.781279 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" event={"ID":"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f","Type":"ContainerDied","Data":"9a50c6347d164eae2e9a1158e12e66f2176d14e59a73438431f11acdd59d3a36"} Jul 22 16:40:46.613566 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:46.613384 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:46.613901 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:46.613421 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:46.613901 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:46.613647 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:46.613901 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:46.613792 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:47.297835 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:47.297808 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:47.297973 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:47.297917 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:47.298021 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:47.297982 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret podName:e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a nodeName:}" failed. No retries permitted until 2026-07-22 16:41:19.297962705 +0000 UTC m=+66.208619860 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret") pod "global-pull-secret-syncer-dmtxl" (UID: "e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:40:47.398415 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:47.398352 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqh8\" (UniqueName: \"kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8\") pod \"network-check-target-jfkr8\" (UID: \"e1f24e3f-ab56-4fa0-951b-45c19ed83baf\") " pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:47.398562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:47.398426 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:47.398562 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:47.398503 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:40:47.398562 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:47.398531 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:40:47.398562 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:47.398544 2581 projected.go:196] Error preparing data for projected volume kube-api-access-hnqh8 for pod openshift-network-diagnostics/network-check-target-jfkr8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:47.398562 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:47.398552 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:47.398807 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:47.398606 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8 podName:e1f24e3f-ab56-4fa0-951b-45c19ed83baf nodeName:}" failed. No retries permitted until 2026-07-22 16:41:19.39858705 +0000 UTC m=+66.309244243 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hnqh8" (UniqueName: "kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8") pod "network-check-target-jfkr8" (UID: "e1f24e3f-ab56-4fa0-951b-45c19ed83baf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:40:47.398807 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:47.398625 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs podName:eaf250c9-d810-4a19-9844-b0e5fa5b8526 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:19.398618847 +0000 UTC m=+66.309276001 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs") pod "network-metrics-daemon-ld4x4" (UID: "eaf250c9-d810-4a19-9844-b0e5fa5b8526") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:40:47.613474 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:47.613407 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:47.613623 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:47.613544 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:48.613471 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:48.613440 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:48.613649 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:48.613488 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:48.613649 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:48.613571 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dmtxl" podUID="e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a" Jul 22 16:40:48.614018 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:48.613690 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfkr8" podUID="e1f24e3f-ab56-4fa0-951b-45c19ed83baf" Jul 22 16:40:49.612693 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.612654 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:49.612906 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:49.612791 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ld4x4" podUID="eaf250c9-d810-4a19-9844-b0e5fa5b8526" Jul 22 16:40:49.919114 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.919093 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeReady" Jul 22 16:40:49.919729 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.919227 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 16:40:49.957197 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.955988 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8"] Jul 22 16:40:49.960988 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.960963 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86675ff59-gknp5"] Jul 22 16:40:49.961133 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.961107 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:49.963321 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.963300 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 16:40:49.963445 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.963427 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-fctvr\"" Jul 22 16:40:49.963763 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.963747 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-55cc957bbb-8brdb"] Jul 22 16:40:49.963851 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.963834 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:49.964483 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.964463 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 16:40:49.964626 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.964584 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 16:40:49.964706 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.964636 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 16:40:49.966718 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.966694 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 16:40:49.966816 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.966737 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 16:40:49.966816 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.966771 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q"] Jul 22 16:40:49.966816 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.966785 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 16:40:49.966816 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.966789 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gtxm5\"" Jul 22 16:40:49.967044 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.967027 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:49.969937 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.969239 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 22 16:40:49.969937 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.969671 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx"] Jul 22 16:40:49.970055 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.970014 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q" Jul 22 16:40:49.970243 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.970223 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 16:40:49.972998 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.971038 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 22 16:40:49.972998 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.971238 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 16:40:49.972998 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.971488 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 22 16:40:49.972998 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.971672 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 22 16:40:49.972998 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.971858 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bwmw7\"" Jul 22 16:40:49.972998 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.972523 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7"] Jul 22 16:40:49.972998 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.972970 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:49.973452 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.973432 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 22 16:40:49.973631 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.973609 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:40:49.974047 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.973941 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mtb6g\"" Jul 22 16:40:49.976061 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.975858 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:40:49.976521 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.976259 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 22 16:40:49.976521 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.976273 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-ltxwv\"" Jul 22 16:40:49.976672 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.976603 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 16:40:49.977048 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.976848 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-m5hgc"] Jul 22 16:40:49.977343 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.977325 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:49.977969 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.977863 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 22 16:40:49.977969 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.977920 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 22 16:40:49.979877 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.979683 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x"] Jul 22 16:40:49.979877 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.979758 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 22 16:40:49.980043 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.979955 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-z2rtj\"" Jul 22 16:40:49.980294 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.980278 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:49.980639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.980469 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 22 16:40:49.981400 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.981360 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:40:49.981613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.981594 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 22 16:40:49.982748 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.982708 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865"] Jul 22 16:40:49.983242 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.983221 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:40:49.985415 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.985022 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:40:49.987277 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.987251 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 22 16:40:49.988982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.988873 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rplml\"" Jul 22 16:40:49.989110 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.989092 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:40:49.991208 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.989460 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 22 16:40:49.991208 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.989536 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dqtbk\"" Jul 22 16:40:49.991208 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.989839 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 22 16:40:49.991208 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.990501 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 22 16:40:49.997049 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.997030 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 22 16:40:49.997417 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.997395 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-lv47m"] Jul 22 16:40:49.998336 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:49.998221 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865" Jul 22 16:40:50.001114 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.001096 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-l8mb2\"" Jul 22 16:40:50.001320 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.001287 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 16:40:50.001479 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.001454 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8"] Jul 22 16:40:50.001599 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.001494 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q"] Jul 22 16:40:50.001599 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.001513 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7"] Jul 22 16:40:50.001599 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.001549 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 16:40:50.001599 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.001577 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx"] Jul 22 16:40:50.001599 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.001597 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86675ff59-gknp5"] Jul 22 16:40:50.001921 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.001617 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hxzdx"] Jul 22 16:40:50.001921 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.001884 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.004450 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.004406 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 22 16:40:50.006705 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.005085 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 16:40:50.006705 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.005177 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rc4wb\"" Jul 22 16:40:50.006705 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.005618 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 16:40:50.006705 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.005818 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4h6hw"] Jul 22 16:40:50.006705 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.005830 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 22 16:40:50.006705 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.005897 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.006705 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.006198 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 22 16:40:50.009639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.008213 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 16:40:50.009639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.008400 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n8ggb\"" Jul 22 16:40:50.009639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.009023 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865"] Jul 22 16:40:50.009639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.009047 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-55cc957bbb-8brdb"] Jul 22 16:40:50.009639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.009060 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x"] Jul 22 16:40:50.009639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.009076 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-m5hgc"] Jul 22 16:40:50.009639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.009094 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hxzdx"] Jul 22 16:40:50.009639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.009112 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-lv47m"] Jul 22 16:40:50.009639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.009176 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4h6hw" Jul 22 16:40:50.010484 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.010388 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 16:40:50.011716 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.011612 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4h6hw"] Jul 22 16:40:50.012670 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.012340 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mkm8j\"" Jul 22 16:40:50.012670 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.012427 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 16:40:50.012670 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.012489 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 22 16:40:50.012670 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.012519 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 16:40:50.012909 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.012729 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 16:40:50.018494 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018474 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-trusted-ca\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.018605 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018505 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2463af8f-0af9-4ff6-ba3b-d944175ebca1-config-volume\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.018605 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018550 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvrs8\" (UniqueName: \"kubernetes.io/projected/170f3511-cd3a-4829-87e2-ffb803cd0b69-kube-api-access-zvrs8\") pod \"volume-data-source-validator-78cf4c6f6-7l82q\" (UID: \"170f3511-cd3a-4829-87e2-ffb803cd0b69\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q" Jul 22 16:40:50.018605 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018588 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.018766 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018614 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh89v\" (UniqueName: \"kubernetes.io/projected/3bde7f11-4339-4d78-8531-b12f8d734aec-kube-api-access-fh89v\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.018766 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018652 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cxt5\" (UniqueName: \"kubernetes.io/projected/a3c3c642-652a-4b5f-b150-04e8b0c7b074-kube-api-access-5cxt5\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:50.018766 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018678 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa257d23-f7ea-498f-a500-b653baad12c8-service-ca-bundle\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.018766 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018702 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.018766 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018725 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2463af8f-0af9-4ff6-ba3b-d944175ebca1-tmp-dir\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.019012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018787 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7967225e-9262-443f-b76a-e532304e06f9-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-8mgm7\" (UID: \"7967225e-9262-443f-b76a-e532304e06f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:50.019012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018827 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.019012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018855 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:50.019012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018881 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dec7161d-014c-45dc-a3ac-9b61d0f1be80-config\") pod \"service-ca-operator-5bcbc94c78-9jmsx\" (UID: \"dec7161d-014c-45dc-a3ac-9b61d0f1be80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:50.019012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018907 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7967225e-9262-443f-b76a-e532304e06f9-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-8mgm7\" (UID: \"7967225e-9262-443f-b76a-e532304e06f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:50.019012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.018975 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fa257d23-f7ea-498f-a500-b653baad12c8-snapshots\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.019276 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019019 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssx4t\" (UniqueName: \"kubernetes.io/projected/dec7161d-014c-45dc-a3ac-9b61d0f1be80-kube-api-access-ssx4t\") pod \"service-ca-operator-5bcbc94c78-9jmsx\" (UID: \"dec7161d-014c-45dc-a3ac-9b61d0f1be80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:50.019276 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019053 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlmq2\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-kube-api-access-xlmq2\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.019276 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019086 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f6fs\" (UniqueName: \"kubernetes.io/projected/981ef2c7-bbdb-4f2e-aeb5-ea05088aa31e-kube-api-access-8f6fs\") pod \"network-check-source-7bc5b6bf85-pg865\" (UID: \"981ef2c7-bbdb-4f2e-aeb5-ea05088aa31e\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865" Jul 22 16:40:50.019276 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019128 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srpwc\" (UniqueName: \"kubernetes.io/projected/7967225e-9262-443f-b76a-e532304e06f9-kube-api-access-srpwc\") pod \"kube-storage-version-migrator-operator-7658d9b749-8mgm7\" (UID: \"7967225e-9262-443f-b76a-e532304e06f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:50.019276 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019169 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a3c3c642-652a-4b5f-b150-04e8b0c7b074-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:50.019276 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019196 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa257d23-f7ea-498f-a500-b653baad12c8-serving-cert\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.019276 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019228 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-installation-pull-secrets\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.019276 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019253 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clg9x\" (UniqueName: \"kubernetes.io/projected/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-kube-api-access-clg9x\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.019608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019292 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:40:50.019608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019326 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.019608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019353 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa257d23-f7ea-498f-a500-b653baad12c8-tmp\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.019608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019397 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-image-registry-private-configuration\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.019608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019448 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bde7f11-4339-4d78-8531-b12f8d734aec-config\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.019608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019471 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bde7f11-4339-4d78-8531-b12f8d734aec-serving-cert\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.019608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019501 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69sx6\" (UniqueName: \"kubernetes.io/projected/fa257d23-f7ea-498f-a500-b653baad12c8-kube-api-access-69sx6\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.019608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019526 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-certificates\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.019608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019561 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-ca-trust-extracted\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.019608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019583 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-bound-sa-token\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.019932 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019611 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-stats-auth\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.019932 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019641 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3bde7f11-4339-4d78-8531-b12f8d734aec-trusted-ca\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.019932 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nrh6\" (UniqueName: \"kubernetes.io/projected/cf0d3172-937e-483a-b1c0-da9606f674a9-kube-api-access-5nrh6\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:40:50.019932 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019697 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dec7161d-014c-45dc-a3ac-9b61d0f1be80-serving-cert\") pod \"service-ca-operator-5bcbc94c78-9jmsx\" (UID: \"dec7161d-014c-45dc-a3ac-9b61d0f1be80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:50.019932 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019724 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-default-certificate\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.019932 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019766 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa257d23-f7ea-498f-a500-b653baad12c8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.019932 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.019793 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnqct\" (UniqueName: \"kubernetes.io/projected/2463af8f-0af9-4ff6-ba3b-d944175ebca1-kube-api-access-nnqct\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.120170 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120140 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bde7f11-4339-4d78-8531-b12f8d734aec-config\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.120282 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120173 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bde7f11-4339-4d78-8531-b12f8d734aec-serving-cert\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.120282 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120202 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69sx6\" (UniqueName: \"kubernetes.io/projected/fa257d23-f7ea-498f-a500-b653baad12c8-kube-api-access-69sx6\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.120282 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120227 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-certificates\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.120282 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120252 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-ca-trust-extracted\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.120282 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-bound-sa-token\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.120562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120295 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-stats-auth\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.120562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120319 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3bde7f11-4339-4d78-8531-b12f8d734aec-trusted-ca\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.120562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120343 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nrh6\" (UniqueName: \"kubernetes.io/projected/cf0d3172-937e-483a-b1c0-da9606f674a9-kube-api-access-5nrh6\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:40:50.120562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120383 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dec7161d-014c-45dc-a3ac-9b61d0f1be80-serving-cert\") pod \"service-ca-operator-5bcbc94c78-9jmsx\" (UID: \"dec7161d-014c-45dc-a3ac-9b61d0f1be80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:50.120562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120409 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-default-certificate\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.120562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120446 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa257d23-f7ea-498f-a500-b653baad12c8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.120562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120470 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnqct\" (UniqueName: \"kubernetes.io/projected/2463af8f-0af9-4ff6-ba3b-d944175ebca1-kube-api-access-nnqct\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.120562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120501 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-trusted-ca\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.120562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120525 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2463af8f-0af9-4ff6-ba3b-d944175ebca1-config-volume\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.120967 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120565 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvrs8\" (UniqueName: \"kubernetes.io/projected/170f3511-cd3a-4829-87e2-ffb803cd0b69-kube-api-access-zvrs8\") pod \"volume-data-source-validator-78cf4c6f6-7l82q\" (UID: \"170f3511-cd3a-4829-87e2-ffb803cd0b69\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q" Jul 22 16:40:50.120967 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120594 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.120967 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120621 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh89v\" (UniqueName: \"kubernetes.io/projected/3bde7f11-4339-4d78-8531-b12f8d734aec-kube-api-access-fh89v\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.120967 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cxt5\" (UniqueName: \"kubernetes.io/projected/a3c3c642-652a-4b5f-b150-04e8b0c7b074-kube-api-access-5cxt5\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:50.120967 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120680 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa257d23-f7ea-498f-a500-b653baad12c8-service-ca-bundle\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.121224 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120962 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-certificates\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.121224 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.121197 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3bde7f11-4339-4d78-8531-b12f8d734aec-trusted-ca\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.121322 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.121235 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa257d23-f7ea-498f-a500-b653baad12c8-service-ca-bundle\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.121394 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.121329 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:40:50.121394 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.121343 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86675ff59-gknp5: secret "image-registry-tls" not found Jul 22 16:40:50.121498 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.121453 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls podName:9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a nodeName:}" failed. No retries permitted until 2026-07-22 16:40:50.62143376 +0000 UTC m=+37.532090924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls") pod "image-registry-86675ff59-gknp5" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a") : secret "image-registry-tls" not found Jul 22 16:40:50.121558 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.121495 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa257d23-f7ea-498f-a500-b653baad12c8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.121955 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.121904 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2463af8f-0af9-4ff6-ba3b-d944175ebca1-config-volume\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.121955 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.120652 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-ca-trust-extracted\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.122107 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.121966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.122107 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.121995 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2463af8f-0af9-4ff6-ba3b-d944175ebca1-tmp-dir\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.122107 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122029 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/191c6bf4-8759-47b4-87ae-077336aa181e-cert\") pod \"ingress-canary-4h6hw\" (UID: \"191c6bf4-8759-47b4-87ae-077336aa181e\") " pod="openshift-ingress-canary/ingress-canary-4h6hw" Jul 22 16:40:50.122107 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122044 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-trusted-ca\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.122107 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122053 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7967225e-9262-443f-b76a-e532304e06f9-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-8mgm7\" (UID: \"7967225e-9262-443f-b76a-e532304e06f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:50.122107 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122098 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122144 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dec7161d-014c-45dc-a3ac-9b61d0f1be80-config\") pod \"service-ca-operator-5bcbc94c78-9jmsx\" (UID: \"dec7161d-014c-45dc-a3ac-9b61d0f1be80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122165 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvdhq\" (UniqueName: \"kubernetes.io/projected/191c6bf4-8759-47b4-87ae-077336aa181e-kube-api-access-dvdhq\") pod \"ingress-canary-4h6hw\" (UID: \"191c6bf4-8759-47b4-87ae-077336aa181e\") " pod="openshift-ingress-canary/ingress-canary-4h6hw" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7967225e-9262-443f-b76a-e532304e06f9-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-8mgm7\" (UID: \"7967225e-9262-443f-b76a-e532304e06f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122211 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fa257d23-f7ea-498f-a500-b653baad12c8-snapshots\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssx4t\" (UniqueName: \"kubernetes.io/projected/dec7161d-014c-45dc-a3ac-9b61d0f1be80-kube-api-access-ssx4t\") pod \"service-ca-operator-5bcbc94c78-9jmsx\" (UID: \"dec7161d-014c-45dc-a3ac-9b61d0f1be80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122248 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2463af8f-0af9-4ff6-ba3b-d944175ebca1-tmp-dir\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122275 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlmq2\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-kube-api-access-xlmq2\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122298 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f6fs\" (UniqueName: \"kubernetes.io/projected/981ef2c7-bbdb-4f2e-aeb5-ea05088aa31e-kube-api-access-8f6fs\") pod \"network-check-source-7bc5b6bf85-pg865\" (UID: \"981ef2c7-bbdb-4f2e-aeb5-ea05088aa31e\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122323 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srpwc\" (UniqueName: \"kubernetes.io/projected/7967225e-9262-443f-b76a-e532304e06f9-kube-api-access-srpwc\") pod \"kube-storage-version-migrator-operator-7658d9b749-8mgm7\" (UID: \"7967225e-9262-443f-b76a-e532304e06f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.122355 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:50.622337456 +0000 UTC m=+37.532994631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : configmap references non-existent config key: service-ca.crt Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122399 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a3c3c642-652a-4b5f-b150-04e8b0c7b074-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:50.122434 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122430 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa257d23-f7ea-498f-a500-b653baad12c8-serving-cert\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122466 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-installation-pull-secrets\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122490 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clg9x\" (UniqueName: \"kubernetes.io/projected/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-kube-api-access-clg9x\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122518 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122542 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122573 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa257d23-f7ea-498f-a500-b653baad12c8-tmp\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122599 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-image-registry-private-configuration\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.122643 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.122711 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:50.622696697 +0000 UTC m=+37.533353861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : secret "router-metrics-certs-default" not found Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.122723 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bde7f11-4339-4d78-8531-b12f8d734aec-config\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.122769 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 16:40:50.123060 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.122799 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls podName:a3c3c642-652a-4b5f-b150-04e8b0c7b074 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:50.622789666 +0000 UTC m=+37.533446833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-h8th8" (UID: "a3c3c642-652a-4b5f-b150-04e8b0c7b074") : secret "cluster-monitoring-operator-tls" not found Jul 22 16:40:50.123616 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.123192 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fa257d23-f7ea-498f-a500-b653baad12c8-snapshots\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.123861 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.123708 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dec7161d-014c-45dc-a3ac-9b61d0f1be80-config\") pod \"service-ca-operator-5bcbc94c78-9jmsx\" (UID: \"dec7161d-014c-45dc-a3ac-9b61d0f1be80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:50.124229 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.124205 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7967225e-9262-443f-b76a-e532304e06f9-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-8mgm7\" (UID: \"7967225e-9262-443f-b76a-e532304e06f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:50.125382 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.125338 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7967225e-9262-443f-b76a-e532304e06f9-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-8mgm7\" (UID: \"7967225e-9262-443f-b76a-e532304e06f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:50.125481 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.125415 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bde7f11-4339-4d78-8531-b12f8d734aec-serving-cert\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.125543 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.125486 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-image-registry-private-configuration\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.125543 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.125523 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 16:40:50.125640 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.125576 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:40:50.125808 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.125580 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls podName:cf0d3172-937e-483a-b1c0-da9606f674a9 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:50.625564139 +0000 UTC m=+37.536221302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8659x" (UID: "cf0d3172-937e-483a-b1c0-da9606f674a9") : secret "samples-operator-tls" not found Jul 22 16:40:50.125919 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.125831 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa257d23-f7ea-498f-a500-b653baad12c8-tmp\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.125919 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.125854 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls podName:2463af8f-0af9-4ff6-ba3b-d944175ebca1 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:50.625835146 +0000 UTC m=+37.536492297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls") pod "dns-default-hxzdx" (UID: "2463af8f-0af9-4ff6-ba3b-d944175ebca1") : secret "dns-default-metrics-tls" not found Jul 22 16:40:50.126163 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.126138 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a3c3c642-652a-4b5f-b150-04e8b0c7b074-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:50.126334 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.126308 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-default-certificate\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.126525 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.126456 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-stats-auth\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.126525 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.126476 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dec7161d-014c-45dc-a3ac-9b61d0f1be80-serving-cert\") pod \"service-ca-operator-5bcbc94c78-9jmsx\" (UID: \"dec7161d-014c-45dc-a3ac-9b61d0f1be80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:50.126802 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.126783 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-installation-pull-secrets\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.128354 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.128333 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-bound-sa-token\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.128428 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.128415 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fa257d23-f7ea-498f-a500-b653baad12c8-serving-cert\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.128601 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.128587 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69sx6\" (UniqueName: \"kubernetes.io/projected/fa257d23-f7ea-498f-a500-b653baad12c8-kube-api-access-69sx6\") pod \"insights-operator-555d9d7885-lv47m\" (UID: \"fa257d23-f7ea-498f-a500-b653baad12c8\") " pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.130312 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.130269 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh89v\" (UniqueName: \"kubernetes.io/projected/3bde7f11-4339-4d78-8531-b12f8d734aec-kube-api-access-fh89v\") pod \"console-operator-565b7b569d-m5hgc\" (UID: \"3bde7f11-4339-4d78-8531-b12f8d734aec\") " pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.131708 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.131687 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvrs8\" (UniqueName: \"kubernetes.io/projected/170f3511-cd3a-4829-87e2-ffb803cd0b69-kube-api-access-zvrs8\") pod \"volume-data-source-validator-78cf4c6f6-7l82q\" (UID: \"170f3511-cd3a-4829-87e2-ffb803cd0b69\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q" Jul 22 16:40:50.131854 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.131806 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cxt5\" (UniqueName: \"kubernetes.io/projected/a3c3c642-652a-4b5f-b150-04e8b0c7b074-kube-api-access-5cxt5\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:50.131854 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.131846 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnqct\" (UniqueName: \"kubernetes.io/projected/2463af8f-0af9-4ff6-ba3b-d944175ebca1-kube-api-access-nnqct\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.132564 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.132543 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nrh6\" (UniqueName: \"kubernetes.io/projected/cf0d3172-937e-483a-b1c0-da9606f674a9-kube-api-access-5nrh6\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:40:50.132904 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.132879 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srpwc\" (UniqueName: \"kubernetes.io/projected/7967225e-9262-443f-b76a-e532304e06f9-kube-api-access-srpwc\") pod \"kube-storage-version-migrator-operator-7658d9b749-8mgm7\" (UID: \"7967225e-9262-443f-b76a-e532304e06f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:50.133177 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.133160 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlmq2\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-kube-api-access-xlmq2\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.133309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.133290 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssx4t\" (UniqueName: \"kubernetes.io/projected/dec7161d-014c-45dc-a3ac-9b61d0f1be80-kube-api-access-ssx4t\") pod \"service-ca-operator-5bcbc94c78-9jmsx\" (UID: \"dec7161d-014c-45dc-a3ac-9b61d0f1be80\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:50.133807 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.133789 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f6fs\" (UniqueName: \"kubernetes.io/projected/981ef2c7-bbdb-4f2e-aeb5-ea05088aa31e-kube-api-access-8f6fs\") pod \"network-check-source-7bc5b6bf85-pg865\" (UID: \"981ef2c7-bbdb-4f2e-aeb5-ea05088aa31e\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865" Jul 22 16:40:50.134795 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.134777 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clg9x\" (UniqueName: \"kubernetes.io/projected/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-kube-api-access-clg9x\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.223551 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.223531 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/191c6bf4-8759-47b4-87ae-077336aa181e-cert\") pod \"ingress-canary-4h6hw\" (UID: \"191c6bf4-8759-47b4-87ae-077336aa181e\") " pod="openshift-ingress-canary/ingress-canary-4h6hw" Jul 22 16:40:50.223630 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.223569 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvdhq\" (UniqueName: \"kubernetes.io/projected/191c6bf4-8759-47b4-87ae-077336aa181e-kube-api-access-dvdhq\") pod \"ingress-canary-4h6hw\" (UID: \"191c6bf4-8759-47b4-87ae-077336aa181e\") " pod="openshift-ingress-canary/ingress-canary-4h6hw" Jul 22 16:40:50.225502 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.225481 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/191c6bf4-8759-47b4-87ae-077336aa181e-cert\") pod \"ingress-canary-4h6hw\" (UID: \"191c6bf4-8759-47b4-87ae-077336aa181e\") " pod="openshift-ingress-canary/ingress-canary-4h6hw" Jul 22 16:40:50.230853 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.230834 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvdhq\" (UniqueName: \"kubernetes.io/projected/191c6bf4-8759-47b4-87ae-077336aa181e-kube-api-access-dvdhq\") pod \"ingress-canary-4h6hw\" (UID: \"191c6bf4-8759-47b4-87ae-077336aa181e\") " pod="openshift-ingress-canary/ingress-canary-4h6hw" Jul 22 16:40:50.353324 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.353296 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q" Jul 22 16:40:50.358409 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.358384 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" Jul 22 16:40:50.363547 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.363531 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" Jul 22 16:40:50.370087 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.370067 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:40:50.380577 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.380556 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865" Jul 22 16:40:50.386653 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.386509 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-lv47m" Jul 22 16:40:50.397206 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.397184 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4h6hw" Jul 22 16:40:50.605469 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.605358 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-m5hgc"] Jul 22 16:40:50.610356 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:50.610322 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bde7f11_4339_4d78_8531_b12f8d734aec.slice/crio-16e2b7730eed1886a7c218435d5f3e0e04e68350ddd2e51602352666f742509b WatchSource:0}: Error finding container 16e2b7730eed1886a7c218435d5f3e0e04e68350ddd2e51602352666f742509b: Status 404 returned error can't find the container with id 16e2b7730eed1886a7c218435d5f3e0e04e68350ddd2e51602352666f742509b Jul 22 16:40:50.611857 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.611832 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx"] Jul 22 16:40:50.611975 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.611919 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q"] Jul 22 16:40:50.612733 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.612643 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:40:50.612876 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.612848 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:40:50.613479 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.613397 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7"] Jul 22 16:40:50.614967 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.614943 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vs4sl\"" Jul 22 16:40:50.615290 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.615084 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 16:40:50.615521 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:50.615487 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddec7161d_014c_45dc_a3ac_9b61d0f1be80.slice/crio-349861c146a279137d5c8a3c4971319259de986dc66d4389118a77fb531ca521 WatchSource:0}: Error finding container 349861c146a279137d5c8a3c4971319259de986dc66d4389118a77fb531ca521: Status 404 returned error can't find the container with id 349861c146a279137d5c8a3c4971319259de986dc66d4389118a77fb531ca521 Jul 22 16:40:50.617314 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:50.616864 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7967225e_9262_443f_b76a_e532304e06f9.slice/crio-6de969d3022ae08ddc469ea205b3135b2d6ee171a06a5081548212d90caa5ce5 WatchSource:0}: Error finding container 6de969d3022ae08ddc469ea205b3135b2d6ee171a06a5081548212d90caa5ce5: Status 404 returned error can't find the container with id 6de969d3022ae08ddc469ea205b3135b2d6ee171a06a5081548212d90caa5ce5 Jul 22 16:40:50.618074 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:50.618037 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170f3511_cd3a_4829_87e2_ffb803cd0b69.slice/crio-8156178b1d7107cb5f12df86551ed897e00c81d27fd59d3f90b195525d3ad191 WatchSource:0}: Error finding container 8156178b1d7107cb5f12df86551ed897e00c81d27fd59d3f90b195525d3ad191: Status 404 returned error can't find the container with id 8156178b1d7107cb5f12df86551ed897e00c81d27fd59d3f90b195525d3ad191 Jul 22 16:40:50.626587 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.626561 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:50.626699 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.626612 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.626699 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.626657 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:50.626699 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.626685 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:50.626699 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626692 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:40:50.626898 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626710 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86675ff59-gknp5: secret "image-registry-tls" not found Jul 22 16:40:50.626898 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.626742 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:40:50.626898 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626760 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls podName:9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a nodeName:}" failed. No retries permitted until 2026-07-22 16:40:51.626742787 +0000 UTC m=+38.537399939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls") pod "image-registry-86675ff59-gknp5" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a") : secret "image-registry-tls" not found Jul 22 16:40:50.626898 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626777 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 16:40:50.626898 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626804 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 16:40:50.626898 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626812 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 16:40:50.626898 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626819 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:51.626800021 +0000 UTC m=+38.537457184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : configmap references non-existent config key: service-ca.crt Jul 22 16:40:50.626898 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626856 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls podName:cf0d3172-937e-483a-b1c0-da9606f674a9 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:51.626842419 +0000 UTC m=+38.537499569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8659x" (UID: "cf0d3172-937e-483a-b1c0-da9606f674a9") : secret "samples-operator-tls" not found Jul 22 16:40:50.626898 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.626877 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:50.627229 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626972 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:51.626961093 +0000 UTC m=+38.537618243 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : secret "router-metrics-certs-default" not found Jul 22 16:40:50.627229 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626976 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:40:50.627229 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.626990 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls podName:a3c3c642-652a-4b5f-b150-04e8b0c7b074 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:51.626980996 +0000 UTC m=+38.537638162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-h8th8" (UID: "a3c3c642-652a-4b5f-b150-04e8b0c7b074") : secret "cluster-monitoring-operator-tls" not found Jul 22 16:40:50.627229 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:50.627024 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls podName:2463af8f-0af9-4ff6-ba3b-d944175ebca1 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:51.627011419 +0000 UTC m=+38.537668571 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls") pod "dns-default-hxzdx" (UID: "2463af8f-0af9-4ff6-ba3b-d944175ebca1") : secret "dns-default-metrics-tls" not found Jul 22 16:40:50.629811 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.629758 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-lv47m"] Jul 22 16:40:50.633509 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:50.633490 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa257d23_f7ea_498f_a500_b653baad12c8.slice/crio-2ef87f7a8e14257c7e9624b25ca43efa71df0daee0adb6862fb1a24ff300940c WatchSource:0}: Error finding container 2ef87f7a8e14257c7e9624b25ca43efa71df0daee0adb6862fb1a24ff300940c: Status 404 returned error can't find the container with id 2ef87f7a8e14257c7e9624b25ca43efa71df0daee0adb6862fb1a24ff300940c Jul 22 16:40:50.636157 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.636142 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4h6hw"] Jul 22 16:40:50.639141 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:50.639123 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191c6bf4_8759_47b4_87ae_077336aa181e.slice/crio-00d4075b3e74d91fc3b52391f0c44fc0b9dda1b301a2cc1fbdb373610aa59013 WatchSource:0}: Error finding container 00d4075b3e74d91fc3b52391f0c44fc0b9dda1b301a2cc1fbdb373610aa59013: Status 404 returned error can't find the container with id 00d4075b3e74d91fc3b52391f0c44fc0b9dda1b301a2cc1fbdb373610aa59013 Jul 22 16:40:50.642032 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.642013 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865"] Jul 22 16:40:50.644929 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:40:50.644912 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod981ef2c7_bbdb_4f2e_aeb5_ea05088aa31e.slice/crio-48b18f5c7a364567cac2105ed44a23c6767692c7fb675957c98502c4b592f841 WatchSource:0}: Error finding container 48b18f5c7a364567cac2105ed44a23c6767692c7fb675957c98502c4b592f841: Status 404 returned error can't find the container with id 48b18f5c7a364567cac2105ed44a23c6767692c7fb675957c98502c4b592f841 Jul 22 16:40:50.794670 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.794643 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4h6hw" event={"ID":"191c6bf4-8759-47b4-87ae-077336aa181e","Type":"ContainerStarted","Data":"00d4075b3e74d91fc3b52391f0c44fc0b9dda1b301a2cc1fbdb373610aa59013"} Jul 22 16:40:50.795687 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.795669 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q" event={"ID":"170f3511-cd3a-4829-87e2-ffb803cd0b69","Type":"ContainerStarted","Data":"8156178b1d7107cb5f12df86551ed897e00c81d27fd59d3f90b195525d3ad191"} Jul 22 16:40:50.796691 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.796673 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" event={"ID":"3bde7f11-4339-4d78-8531-b12f8d734aec","Type":"ContainerStarted","Data":"16e2b7730eed1886a7c218435d5f3e0e04e68350ddd2e51602352666f742509b"} Jul 22 16:40:50.799720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.799695 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" event={"ID":"dec7161d-014c-45dc-a3ac-9b61d0f1be80","Type":"ContainerStarted","Data":"349861c146a279137d5c8a3c4971319259de986dc66d4389118a77fb531ca521"} Jul 22 16:40:50.804086 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.801264 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865" event={"ID":"981ef2c7-bbdb-4f2e-aeb5-ea05088aa31e","Type":"ContainerStarted","Data":"48b18f5c7a364567cac2105ed44a23c6767692c7fb675957c98502c4b592f841"} Jul 22 16:40:50.806707 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.806687 2581 generic.go:356] "Generic (PLEG): container finished" podID="e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f" containerID="c379a3353642ea14f2cd7e6ee1f5a94f942fe9f2ae815bfb11e226944f51533d" exitCode=0 Jul 22 16:40:50.806786 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.806740 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" event={"ID":"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f","Type":"ContainerDied","Data":"c379a3353642ea14f2cd7e6ee1f5a94f942fe9f2ae815bfb11e226944f51533d"} Jul 22 16:40:50.807833 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.807812 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" event={"ID":"7967225e-9262-443f-b76a-e532304e06f9","Type":"ContainerStarted","Data":"6de969d3022ae08ddc469ea205b3135b2d6ee171a06a5081548212d90caa5ce5"} Jul 22 16:40:50.808776 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:50.808751 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lv47m" event={"ID":"fa257d23-f7ea-498f-a500-b653baad12c8","Type":"ContainerStarted","Data":"2ef87f7a8e14257c7e9624b25ca43efa71df0daee0adb6862fb1a24ff300940c"} Jul 22 16:40:51.617483 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.617452 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:40:51.622066 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.622038 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 16:40:51.622348 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.622331 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d5r55\"" Jul 22 16:40:51.635882 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.635828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:51.635882 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.635870 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:51.636036 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.635894 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:51.636036 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.635910 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:51.636036 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.635938 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:40:51.636036 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.635953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:40:51.636036 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.635957 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86675ff59-gknp5: secret "image-registry-tls" not found Jul 22 16:40:51.636036 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.635970 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:51.636036 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.636017 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls podName:9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a nodeName:}" failed. No retries permitted until 2026-07-22 16:40:53.63599766 +0000 UTC m=+40.546654823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls") pod "image-registry-86675ff59-gknp5" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a") : secret "image-registry-tls" not found Jul 22 16:40:51.636394 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.636053 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:40:51.636394 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.636095 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls podName:2463af8f-0af9-4ff6-ba3b-d944175ebca1 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:53.636084569 +0000 UTC m=+40.546741719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls") pod "dns-default-hxzdx" (UID: "2463af8f-0af9-4ff6-ba3b-d944175ebca1") : secret "dns-default-metrics-tls" not found Jul 22 16:40:51.636394 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.636143 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:53.636136782 +0000 UTC m=+40.546793931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : configmap references non-existent config key: service-ca.crt Jul 22 16:40:51.636394 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.636194 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 16:40:51.636394 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.636218 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:53.636210581 +0000 UTC m=+40.546867731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : secret "router-metrics-certs-default" not found Jul 22 16:40:51.636394 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.636272 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 16:40:51.636394 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.636302 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls podName:a3c3c642-652a-4b5f-b150-04e8b0c7b074 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:53.636293122 +0000 UTC m=+40.546950275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-h8th8" (UID: "a3c3c642-652a-4b5f-b150-04e8b0c7b074") : secret "cluster-monitoring-operator-tls" not found Jul 22 16:40:51.636394 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.636348 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 16:40:51.636786 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:51.636413 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls podName:cf0d3172-937e-483a-b1c0-da9606f674a9 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:53.636399069 +0000 UTC m=+40.547056223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8659x" (UID: "cf0d3172-937e-483a-b1c0-da9606f674a9") : secret "samples-operator-tls" not found Jul 22 16:40:51.820977 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.820941 2581 generic.go:356] "Generic (PLEG): container finished" podID="e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f" containerID="e324f5f776dc24ecd5251518eeea4397940a89fe1e586315719fe6b85593d57d" exitCode=0 Jul 22 16:40:51.821140 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:51.821009 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" event={"ID":"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f","Type":"ContainerDied","Data":"e324f5f776dc24ecd5251518eeea4397940a89fe1e586315719fe6b85593d57d"} Jul 22 16:40:52.828705 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:52.828468 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q" event={"ID":"170f3511-cd3a-4829-87e2-ffb803cd0b69","Type":"ContainerStarted","Data":"6aecd67cfd11b0f161806f6baef28cd7e713669f8f6e8fdd35b8fb5befdf9a42"} Jul 22 16:40:52.835206 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:52.835178 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" event={"ID":"e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f","Type":"ContainerStarted","Data":"698706b15f7dac326d8eb3e4366d6757c03ef5baa5bf276640ffb21cad708c0a"} Jul 22 16:40:52.848727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:52.847893 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-7l82q" podStartSLOduration=16.185385758 podStartE2EDuration="17.847874817s" totalImagesPullingTime="1.662489059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:52.846295139 +0000 UTC m=+39.756952315" watchObservedRunningTime="2026-07-22 16:40:52.847874817 +0000 UTC m=+39.758531995" Jul 22 16:40:52.879737 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:52.878816 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8xzv4" podStartSLOduration=7.698716489 podStartE2EDuration="38.873848349s" totalImagesPullingTime="31.17513186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:52.872116947 +0000 UTC m=+39.782774121" watchObservedRunningTime="2026-07-22 16:40:52.873848349 +0000 UTC m=+39.784505523" Jul 22 16:40:53.659088 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:53.659050 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:40:53.659088 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:53.659094 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:53.659297 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:53.659196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:53.659297 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:53.659231 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:53.659297 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:53.659262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:53.659297 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:53.659289 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659292 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659359 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls podName:2463af8f-0af9-4ff6-ba3b-d944175ebca1 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:57.659339307 +0000 UTC m=+44.569996468 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls") pod "dns-default-hxzdx" (UID: "2463af8f-0af9-4ff6-ba3b-d944175ebca1") : secret "dns-default-metrics-tls" not found Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659359 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659393 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:57.659382594 +0000 UTC m=+44.570039744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : configmap references non-existent config key: service-ca.crt Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659400 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86675ff59-gknp5: secret "image-registry-tls" not found Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659384 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659424 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659441 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls podName:a3c3c642-652a-4b5f-b150-04e8b0c7b074 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:57.659431147 +0000 UTC m=+44.570088299 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-h8th8" (UID: "a3c3c642-652a-4b5f-b150-04e8b0c7b074") : secret "cluster-monitoring-operator-tls" not found Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659292 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659459 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:57.659448762 +0000 UTC m=+44.570105915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : secret "router-metrics-certs-default" not found Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659475 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls podName:9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a nodeName:}" failed. No retries permitted until 2026-07-22 16:40:57.659466909 +0000 UTC m=+44.570124060 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls") pod "image-registry-86675ff59-gknp5" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a") : secret "image-registry-tls" not found Jul 22 16:40:53.659522 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:53.659487 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls podName:cf0d3172-937e-483a-b1c0-da9606f674a9 nodeName:}" failed. No retries permitted until 2026-07-22 16:40:57.659481658 +0000 UTC m=+44.570138807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8659x" (UID: "cf0d3172-937e-483a-b1c0-da9606f674a9") : secret "samples-operator-tls" not found Jul 22 16:40:57.694826 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:57.694788 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:40:57.694826 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:57.694831 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:57.694851 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:57.694867 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:57.694912 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:57.694930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.694974 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.694984 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.695005 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86675ff59-gknp5: secret "image-registry-tls" not found Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.695010 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.695051 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:05.695033804 +0000 UTC m=+52.605690954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : secret "router-metrics-certs-default" not found Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.695057 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.695069 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls podName:9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a nodeName:}" failed. No retries permitted until 2026-07-22 16:41:05.695060127 +0000 UTC m=+52.605717279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls") pod "image-registry-86675ff59-gknp5" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a") : secret "image-registry-tls" not found Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.695084 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls podName:a3c3c642-652a-4b5f-b150-04e8b0c7b074 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:05.695076617 +0000 UTC m=+52.605733770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-h8th8" (UID: "a3c3c642-652a-4b5f-b150-04e8b0c7b074") : secret "cluster-monitoring-operator-tls" not found Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.695113 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls podName:cf0d3172-937e-483a-b1c0-da9606f674a9 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:05.695100684 +0000 UTC m=+52.605757834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8659x" (UID: "cf0d3172-937e-483a-b1c0-da9606f674a9") : secret "samples-operator-tls" not found Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.695059 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:40:57.695491 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.695139 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:05.695123565 +0000 UTC m=+52.605780731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : configmap references non-existent config key: service-ca.crt Jul 22 16:40:57.695917 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:57.695158 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls podName:2463af8f-0af9-4ff6-ba3b-d944175ebca1 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:05.695150986 +0000 UTC m=+52.605808135 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls") pod "dns-default-hxzdx" (UID: "2463af8f-0af9-4ff6-ba3b-d944175ebca1") : secret "dns-default-metrics-tls" not found Jul 22 16:40:58.852504 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.852437 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" event={"ID":"7967225e-9262-443f-b76a-e532304e06f9","Type":"ContainerStarted","Data":"82c81b4ccf77a1b47d90da1960c78e058a908d8fb78889f24ed411a048e6581d"} Jul 22 16:40:58.854122 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.854094 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lv47m" event={"ID":"fa257d23-f7ea-498f-a500-b653baad12c8","Type":"ContainerStarted","Data":"62ded8fb6509578eb5fa1e70d821b0eec5547c10f6764d791643de0c13debb20"} Jul 22 16:40:58.855859 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.855808 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4h6hw" event={"ID":"191c6bf4-8759-47b4-87ae-077336aa181e","Type":"ContainerStarted","Data":"752ec467b3fea136fc8b418f0e52ae3d9479fc1f11b7942795616769bd876dc7"} Jul 22 16:40:58.857856 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.857835 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/0.log" Jul 22 16:40:58.857969 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.857875 2581 generic.go:356] "Generic (PLEG): container finished" podID="3bde7f11-4339-4d78-8531-b12f8d734aec" containerID="3e7a160bf516c429a67f9264e338188f7ab430473797cdd591f1b51cd9c02f93" exitCode=255 Jul 22 16:40:58.857969 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.857940 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" event={"ID":"3bde7f11-4339-4d78-8531-b12f8d734aec","Type":"ContainerDied","Data":"3e7a160bf516c429a67f9264e338188f7ab430473797cdd591f1b51cd9c02f93"} Jul 22 16:40:58.858219 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.858200 2581 scope.go:117] "RemoveContainer" containerID="3e7a160bf516c429a67f9264e338188f7ab430473797cdd591f1b51cd9c02f93" Jul 22 16:40:58.859786 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.859759 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" event={"ID":"dec7161d-014c-45dc-a3ac-9b61d0f1be80","Type":"ContainerStarted","Data":"5921720096b12946b5100c84882968d2533a7fdab20fc4681e8d043d0bbe9458"} Jul 22 16:40:58.861717 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.861700 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865" event={"ID":"981ef2c7-bbdb-4f2e-aeb5-ea05088aa31e","Type":"ContainerStarted","Data":"cd3f6698e0a84e3fe95b6fc78a61d2677842d061797d1545a0f7c4a5ff20e6bf"} Jul 22 16:40:58.872348 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.872305 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" podStartSLOduration=17.300913449 podStartE2EDuration="24.872288328s" totalImagesPullingTime="7.571374879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:58.870973805 +0000 UTC m=+45.781630977" watchObservedRunningTime="2026-07-22 16:40:58.872288328 +0000 UTC m=+45.782945503" Jul 22 16:40:58.899313 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.899269 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-pg865" podStartSLOduration=6.35444809 podStartE2EDuration="13.899250486s" totalImagesPullingTime="7.544802396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:58.898182904 +0000 UTC m=+45.808840077" watchObservedRunningTime="2026-07-22 16:40:58.899250486 +0000 UTC m=+45.809907658" Jul 22 16:40:58.943946 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.943906 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4h6hw" podStartSLOduration=2.393338214 podStartE2EDuration="9.943891246s" totalImagesPullingTime="7.550553032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:58.917156983 +0000 UTC m=+45.827814156" watchObservedRunningTime="2026-07-22 16:40:58.943891246 +0000 UTC m=+45.854548417" Jul 22 16:40:58.944348 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.944304 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-lv47m" podStartSLOduration=26.388897522 podStartE2EDuration="33.944294029s" totalImagesPullingTime="7.555396507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:58.943195173 +0000 UTC m=+45.853852346" watchObservedRunningTime="2026-07-22 16:40:58.944294029 +0000 UTC m=+45.854951201" Jul 22 16:40:58.991027 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:58.989347 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" podStartSLOduration=16.410530223 podStartE2EDuration="23.989325527s" totalImagesPullingTime="7.578795304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:58.968249209 +0000 UTC m=+45.878906381" watchObservedRunningTime="2026-07-22 16:40:58.989325527 +0000 UTC m=+45.899982700" Jul 22 16:40:59.762567 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.762534 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gfzrg"] Jul 22 16:40:59.788068 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.788043 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gfzrg"] Jul 22 16:40:59.788234 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.788183 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:40:59.790444 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.790403 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 16:40:59.790601 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.790513 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 16:40:59.790743 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.790600 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hs6zh\"" Jul 22 16:40:59.867200 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.867179 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/1.log" Jul 22 16:40:59.867601 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.867588 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/0.log" Jul 22 16:40:59.867654 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.867620 2581 generic.go:356] "Generic (PLEG): container finished" podID="3bde7f11-4339-4d78-8531-b12f8d734aec" containerID="597c54cfce552fdc6db7427059b5bce25c2f91f26e9ad5d10c5c11d09fcf4456" exitCode=255 Jul 22 16:40:59.867755 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.867730 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" event={"ID":"3bde7f11-4339-4d78-8531-b12f8d734aec","Type":"ContainerDied","Data":"597c54cfce552fdc6db7427059b5bce25c2f91f26e9ad5d10c5c11d09fcf4456"} Jul 22 16:40:59.867815 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.867786 2581 scope.go:117] "RemoveContainer" containerID="3e7a160bf516c429a67f9264e338188f7ab430473797cdd591f1b51cd9c02f93" Jul 22 16:40:59.867928 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.867907 2581 scope.go:117] "RemoveContainer" containerID="597c54cfce552fdc6db7427059b5bce25c2f91f26e9ad5d10c5c11d09fcf4456" Jul 22 16:40:59.868108 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:40:59.868088 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-m5hgc_openshift-console-operator(3bde7f11-4339-4d78-8531-b12f8d734aec)\"" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" podUID="3bde7f11-4339-4d78-8531-b12f8d734aec" Jul 22 16:40:59.914419 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.914397 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlgc6\" (UniqueName: \"kubernetes.io/projected/a1409262-765e-45c5-97b1-df28ad5043ed-kube-api-access-jlgc6\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:40:59.914536 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.914487 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a1409262-765e-45c5-97b1-df28ad5043ed-crio-socket\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:40:59.914990 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.914964 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:40:59.915102 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.915071 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1409262-765e-45c5-97b1-df28ad5043ed-data-volume\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:40:59.915208 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:40:59.915192 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.016332 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.016256 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.016332 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.016300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1409262-765e-45c5-97b1-df28ad5043ed-data-volume\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.016534 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.016346 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.016534 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.016383 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlgc6\" (UniqueName: \"kubernetes.io/projected/a1409262-765e-45c5-97b1-df28ad5043ed-kube-api-access-jlgc6\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.016534 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.016406 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a1409262-765e-45c5-97b1-df28ad5043ed-crio-socket\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.016534 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:00.016432 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:41:00.016675 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:00.016538 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls podName:a1409262-765e-45c5-97b1-df28ad5043ed nodeName:}" failed. No retries permitted until 2026-07-22 16:41:00.516518855 +0000 UTC m=+47.427176012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfzrg" (UID: "a1409262-765e-45c5-97b1-df28ad5043ed") : secret "insights-runtime-extractor-tls" not found Jul 22 16:41:00.016675 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.016598 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a1409262-765e-45c5-97b1-df28ad5043ed-crio-socket\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.016675 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:00.016608 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:41:00.016675 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:00.016663 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm podName:a1409262-765e-45c5-97b1-df28ad5043ed nodeName:}" failed. No retries permitted until 2026-07-22 16:41:00.516647617 +0000 UTC m=+47.427304781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfzrg" (UID: "a1409262-765e-45c5-97b1-df28ad5043ed") : configmap "kube-rbac-proxy" not found Jul 22 16:41:00.016831 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.016724 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1409262-765e-45c5-97b1-df28ad5043ed-data-volume\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.029484 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.029445 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlgc6\" (UniqueName: \"kubernetes.io/projected/a1409262-765e-45c5-97b1-df28ad5043ed-kube-api-access-jlgc6\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.370864 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.370788 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:41:00.370864 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.370845 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:41:00.520791 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.520761 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.520982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.520836 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:00.520982 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:00.520936 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:41:00.520982 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:00.520970 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:41:00.521143 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:00.521014 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls podName:a1409262-765e-45c5-97b1-df28ad5043ed nodeName:}" failed. No retries permitted until 2026-07-22 16:41:01.520993771 +0000 UTC m=+48.431650926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfzrg" (UID: "a1409262-765e-45c5-97b1-df28ad5043ed") : secret "insights-runtime-extractor-tls" not found Jul 22 16:41:00.521143 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:00.521034 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm podName:a1409262-765e-45c5-97b1-df28ad5043ed nodeName:}" failed. No retries permitted until 2026-07-22 16:41:01.521024588 +0000 UTC m=+48.431681741 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfzrg" (UID: "a1409262-765e-45c5-97b1-df28ad5043ed") : configmap "kube-rbac-proxy" not found Jul 22 16:41:00.871598 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.871572 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/1.log" Jul 22 16:41:00.872019 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:00.871885 2581 scope.go:117] "RemoveContainer" containerID="597c54cfce552fdc6db7427059b5bce25c2f91f26e9ad5d10c5c11d09fcf4456" Jul 22 16:41:00.872076 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:00.872026 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-m5hgc_openshift-console-operator(3bde7f11-4339-4d78-8531-b12f8d734aec)\"" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" podUID="3bde7f11-4339-4d78-8531-b12f8d734aec" Jul 22 16:41:01.529217 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:01.529179 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:01.529409 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:01.529279 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:01.529409 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:01.529335 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:41:01.529525 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:01.529416 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls podName:a1409262-765e-45c5-97b1-df28ad5043ed nodeName:}" failed. No retries permitted until 2026-07-22 16:41:03.529400722 +0000 UTC m=+50.440057882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfzrg" (UID: "a1409262-765e-45c5-97b1-df28ad5043ed") : secret "insights-runtime-extractor-tls" not found Jul 22 16:41:01.529525 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:01.529423 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:41:01.529525 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:01.529487 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm podName:a1409262-765e-45c5-97b1-df28ad5043ed nodeName:}" failed. No retries permitted until 2026-07-22 16:41:03.529469319 +0000 UTC m=+50.440126474 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfzrg" (UID: "a1409262-765e-45c5-97b1-df28ad5043ed") : configmap "kube-rbac-proxy" not found Jul 22 16:41:01.874852 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:01.874786 2581 scope.go:117] "RemoveContainer" containerID="597c54cfce552fdc6db7427059b5bce25c2f91f26e9ad5d10c5c11d09fcf4456" Jul 22 16:41:01.875216 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:01.874926 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-m5hgc_openshift-console-operator(3bde7f11-4339-4d78-8531-b12f8d734aec)\"" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" podUID="3bde7f11-4339-4d78-8531-b12f8d734aec" Jul 22 16:41:01.945504 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:01.945478 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7tdk9_d1de09de-782e-4b78-a68d-cbf1078cbc90/dns-node-resolver/0.log" Jul 22 16:41:03.146096 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:03.146070 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nxfwf_b77686d0-8cd0-4908-98fb-f34ff9a63c79/node-ca/0.log" Jul 22 16:41:03.546326 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:03.546289 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:03.546529 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:03.546384 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:03.546529 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:03.546456 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:41:03.546529 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:03.546458 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:41:03.546529 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:03.546509 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm podName:a1409262-765e-45c5-97b1-df28ad5043ed nodeName:}" failed. No retries permitted until 2026-07-22 16:41:07.546494292 +0000 UTC m=+54.457151441 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfzrg" (UID: "a1409262-765e-45c5-97b1-df28ad5043ed") : configmap "kube-rbac-proxy" not found Jul 22 16:41:03.546529 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:03.546522 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls podName:a1409262-765e-45c5-97b1-df28ad5043ed nodeName:}" failed. No retries permitted until 2026-07-22 16:41:07.546516137 +0000 UTC m=+54.457173287 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfzrg" (UID: "a1409262-765e-45c5-97b1-df28ad5043ed") : secret "insights-runtime-extractor-tls" not found Jul 22 16:41:03.557889 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:03.557871 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4h6hw_191c6bf4-8759-47b4-87ae-077336aa181e/serve-healthcheck-canary/0.log" Jul 22 16:41:04.348867 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:04.348811 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-8mgm7_7967225e-9262-443f-b76a-e532304e06f9/kube-storage-version-migrator-operator/0.log" Jul 22 16:41:05.762929 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:05.762900 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:05.762935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:05.762957 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:05.762977 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763038 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763054 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86675ff59-gknp5: secret "image-registry-tls" not found Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763069 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763078 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:21.763057166 +0000 UTC m=+68.673714318 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : configmap references non-existent config key: service-ca.crt Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:05.763039 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763108 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls podName:9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a nodeName:}" failed. No retries permitted until 2026-07-22 16:41:21.763096757 +0000 UTC m=+68.673753908 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls") pod "image-registry-86675ff59-gknp5" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a") : secret "image-registry-tls" not found Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763108 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763160 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763213 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs podName:f88cc0d4-8c3d-4c35-9326-ce054e6cf331 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:21.763202141 +0000 UTC m=+68.673859302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs") pod "router-default-55cc957bbb-8brdb" (UID: "f88cc0d4-8c3d-4c35-9326-ce054e6cf331") : secret "router-metrics-certs-default" not found Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763228 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls podName:cf0d3172-937e-483a-b1c0-da9606f674a9 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:21.763222521 +0000 UTC m=+68.673879674 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-8659x" (UID: "cf0d3172-937e-483a-b1c0-da9606f674a9") : secret "samples-operator-tls" not found Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:05.763245 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763287 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:41:05.763301 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763297 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls podName:a3c3c642-652a-4b5f-b150-04e8b0c7b074 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:21.763291645 +0000 UTC m=+68.673948794 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-h8th8" (UID: "a3c3c642-652a-4b5f-b150-04e8b0c7b074") : secret "cluster-monitoring-operator-tls" not found Jul 22 16:41:05.763824 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:05.763318 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls podName:2463af8f-0af9-4ff6-ba3b-d944175ebca1 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:21.763307993 +0000 UTC m=+68.673965143 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls") pod "dns-default-hxzdx" (UID: "2463af8f-0af9-4ff6-ba3b-d944175ebca1") : secret "dns-default-metrics-tls" not found Jul 22 16:41:07.577659 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:07.577626 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:07.578191 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:07.577694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:07.578191 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:07.577792 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:41:07.578191 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:07.577868 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls podName:a1409262-765e-45c5-97b1-df28ad5043ed nodeName:}" failed. No retries permitted until 2026-07-22 16:41:15.577848287 +0000 UTC m=+62.488505442 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfzrg" (UID: "a1409262-765e-45c5-97b1-df28ad5043ed") : secret "insights-runtime-extractor-tls" not found Jul 22 16:41:07.578191 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:07.578179 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a1409262-765e-45c5-97b1-df28ad5043ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:14.795707 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:14.795680 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dsxzr" Jul 22 16:41:15.613448 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.613416 2581 scope.go:117] "RemoveContainer" containerID="597c54cfce552fdc6db7427059b5bce25c2f91f26e9ad5d10c5c11d09fcf4456" Jul 22 16:41:15.645933 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.645908 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:15.648223 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.648192 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a1409262-765e-45c5-97b1-df28ad5043ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfzrg\" (UID: \"a1409262-765e-45c5-97b1-df28ad5043ed\") " pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:15.700719 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.700599 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hs6zh\"" Jul 22 16:41:15.708161 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.708139 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gfzrg" Jul 22 16:41:15.825428 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.825402 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gfzrg"] Jul 22 16:41:15.828529 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:15.828495 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1409262_765e_45c5_97b1_df28ad5043ed.slice/crio-c538463abf7e824398772df09234b94cf40cfd12a01851055014e6b7ac3523d8 WatchSource:0}: Error finding container c538463abf7e824398772df09234b94cf40cfd12a01851055014e6b7ac3523d8: Status 404 returned error can't find the container with id c538463abf7e824398772df09234b94cf40cfd12a01851055014e6b7ac3523d8 Jul 22 16:41:15.917087 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.917033 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 16:41:15.917431 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.917414 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/1.log" Jul 22 16:41:15.917521 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.917451 2581 generic.go:356] "Generic (PLEG): container finished" podID="3bde7f11-4339-4d78-8531-b12f8d734aec" containerID="8ebe7c0964cbbd7470bd9d2b3df643b4fe351509688fcb1483b9011ec1c47298" exitCode=255 Jul 22 16:41:15.917576 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.917518 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" event={"ID":"3bde7f11-4339-4d78-8531-b12f8d734aec","Type":"ContainerDied","Data":"8ebe7c0964cbbd7470bd9d2b3df643b4fe351509688fcb1483b9011ec1c47298"} Jul 22 16:41:15.917618 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.917597 2581 scope.go:117] "RemoveContainer" containerID="597c54cfce552fdc6db7427059b5bce25c2f91f26e9ad5d10c5c11d09fcf4456" Jul 22 16:41:15.917950 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.917931 2581 scope.go:117] "RemoveContainer" containerID="8ebe7c0964cbbd7470bd9d2b3df643b4fe351509688fcb1483b9011ec1c47298" Jul 22 16:41:15.918124 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:15.918102 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-m5hgc_openshift-console-operator(3bde7f11-4339-4d78-8531-b12f8d734aec)\"" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" podUID="3bde7f11-4339-4d78-8531-b12f8d734aec" Jul 22 16:41:15.918720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:15.918702 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfzrg" event={"ID":"a1409262-765e-45c5-97b1-df28ad5043ed","Type":"ContainerStarted","Data":"c538463abf7e824398772df09234b94cf40cfd12a01851055014e6b7ac3523d8"} Jul 22 16:41:16.924263 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:16.924233 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 16:41:19.379547 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.379508 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:41:19.383743 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.383708 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 16:41:19.393114 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.393092 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a-original-pull-secret\") pod \"global-pull-secret-syncer-dmtxl\" (UID: \"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a\") " pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:41:19.433399 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.433355 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dmtxl" Jul 22 16:41:19.481028 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.481005 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqh8\" (UniqueName: \"kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8\") pod \"network-check-target-jfkr8\" (UID: \"e1f24e3f-ab56-4fa0-951b-45c19ed83baf\") " pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:41:19.481146 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.481119 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:41:19.483845 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.483825 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnqh8\" (UniqueName: \"kubernetes.io/projected/e1f24e3f-ab56-4fa0-951b-45c19ed83baf-kube-api-access-hnqh8\") pod \"network-check-target-jfkr8\" (UID: \"e1f24e3f-ab56-4fa0-951b-45c19ed83baf\") " pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:41:19.484919 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.484899 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 16:41:19.494310 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.494289 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eaf250c9-d810-4a19-9844-b0e5fa5b8526-metrics-certs\") pod \"network-metrics-daemon-ld4x4\" (UID: \"eaf250c9-d810-4a19-9844-b0e5fa5b8526\") " pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:41:19.533873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.533850 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d5r55\"" Jul 22 16:41:19.541971 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.541947 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ld4x4" Jul 22 16:41:19.729272 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.729242 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vs4sl\"" Jul 22 16:41:19.737357 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:19.737330 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:41:20.371192 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.371156 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:41:20.371353 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.371205 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:41:20.371558 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.371543 2581 scope.go:117] "RemoveContainer" containerID="8ebe7c0964cbbd7470bd9d2b3df643b4fe351509688fcb1483b9011ec1c47298" Jul 22 16:41:20.371759 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:20.371737 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-m5hgc_openshift-console-operator(3bde7f11-4339-4d78-8531-b12f8d734aec)\"" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" podUID="3bde7f11-4339-4d78-8531-b12f8d734aec" Jul 22 16:41:20.706517 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.706443 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jfkr8"] Jul 22 16:41:20.708289 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:20.708263 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f24e3f_ab56_4fa0_951b_45c19ed83baf.slice/crio-598da17ca6ffd7fc00a1335262dde01125c39f2036b952b2f0b487a5ed4060e1 WatchSource:0}: Error finding container 598da17ca6ffd7fc00a1335262dde01125c39f2036b952b2f0b487a5ed4060e1: Status 404 returned error can't find the container with id 598da17ca6ffd7fc00a1335262dde01125c39f2036b952b2f0b487a5ed4060e1 Jul 22 16:41:20.919657 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.919587 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dmtxl"] Jul 22 16:41:20.922696 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.922666 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ld4x4"] Jul 22 16:41:20.922960 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:20.922917 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0eb8cbf_7a75_4b2e_8bf4_9439e5b8975a.slice/crio-e466d93caa07d6d5e448c71a90fd51d6550d8103e053ad734c2323c7e848bdc0 WatchSource:0}: Error finding container e466d93caa07d6d5e448c71a90fd51d6550d8103e053ad734c2323c7e848bdc0: Status 404 returned error can't find the container with id e466d93caa07d6d5e448c71a90fd51d6550d8103e053ad734c2323c7e848bdc0 Jul 22 16:41:20.925301 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:20.925285 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf250c9_d810_4a19_9844_b0e5fa5b8526.slice/crio-ec3515b220e98401098304dc91b5e29d4b5eb929f586ec890090f9f685c70627 WatchSource:0}: Error finding container ec3515b220e98401098304dc91b5e29d4b5eb929f586ec890090f9f685c70627: Status 404 returned error can't find the container with id ec3515b220e98401098304dc91b5e29d4b5eb929f586ec890090f9f685c70627 Jul 22 16:41:20.940855 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.940827 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfzrg" event={"ID":"a1409262-765e-45c5-97b1-df28ad5043ed","Type":"ContainerStarted","Data":"6e73c5ddd199c000dfd50c94d507e677de759215fbda887e702a8aaba69a9160"} Jul 22 16:41:20.941904 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.941880 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dmtxl" event={"ID":"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a","Type":"ContainerStarted","Data":"e466d93caa07d6d5e448c71a90fd51d6550d8103e053ad734c2323c7e848bdc0"} Jul 22 16:41:20.942932 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.942913 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ld4x4" event={"ID":"eaf250c9-d810-4a19-9844-b0e5fa5b8526","Type":"ContainerStarted","Data":"ec3515b220e98401098304dc91b5e29d4b5eb929f586ec890090f9f685c70627"} Jul 22 16:41:20.944053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.944033 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jfkr8" event={"ID":"e1f24e3f-ab56-4fa0-951b-45c19ed83baf","Type":"ContainerStarted","Data":"0d124b9a2165662b59b64d334f3400a574e8d3a369421c96473b59cc3c79ef6e"} Jul 22 16:41:20.944116 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.944067 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jfkr8" event={"ID":"e1f24e3f-ab56-4fa0-951b-45c19ed83baf","Type":"ContainerStarted","Data":"598da17ca6ffd7fc00a1335262dde01125c39f2036b952b2f0b487a5ed4060e1"} Jul 22 16:41:20.944178 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.944162 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:41:20.960453 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:20.960419 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jfkr8" podStartSLOduration=66.960407318 podStartE2EDuration="1m6.960407318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:20.960018981 +0000 UTC m=+67.870676154" watchObservedRunningTime="2026-07-22 16:41:20.960407318 +0000 UTC m=+67.871064468" Jul 22 16:41:21.800201 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.800162 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:21.800660 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.800219 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:21.800660 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.800251 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:41:21.800660 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.800302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:41:21.800660 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.800328 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:41:21.800660 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.800429 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:41:21.802508 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.801867 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-service-ca-bundle\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:21.812509 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.810829 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2463af8f-0af9-4ff6-ba3b-d944175ebca1-metrics-tls\") pod \"dns-default-hxzdx\" (UID: \"2463af8f-0af9-4ff6-ba3b-d944175ebca1\") " pod="openshift-dns/dns-default-hxzdx" Jul 22 16:41:21.812509 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.811961 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf0d3172-937e-483a-b1c0-da9606f674a9-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-8659x\" (UID: \"cf0d3172-937e-483a-b1c0-da9606f674a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:41:21.812689 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.812674 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3c3c642-652a-4b5f-b150-04e8b0c7b074-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-h8th8\" (UID: \"a3c3c642-652a-4b5f-b150-04e8b0c7b074\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:41:21.813163 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.813123 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88cc0d4-8c3d-4c35-9326-ce054e6cf331-metrics-certs\") pod \"router-default-55cc957bbb-8brdb\" (UID: \"f88cc0d4-8c3d-4c35-9326-ce054e6cf331\") " pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:21.813289 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.813270 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") pod \"image-registry-86675ff59-gknp5\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:41:21.879564 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.879191 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rplml\"" Jul 22 16:41:21.885439 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.885412 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" Jul 22 16:41:21.895316 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.895292 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n8ggb\"" Jul 22 16:41:21.903499 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:21.903465 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hxzdx" Jul 22 16:41:22.078188 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.078106 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-fctvr\"" Jul 22 16:41:22.085500 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.085474 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" Jul 22 16:41:22.093345 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.092214 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gtxm5\"" Jul 22 16:41:22.099459 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.099438 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:41:22.101124 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.101103 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bwmw7\"" Jul 22 16:41:22.109557 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.109465 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:22.478264 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.477556 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hxzdx"] Jul 22 16:41:22.661325 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:22.661251 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2463af8f_0af9_4ff6_ba3b_d944175ebca1.slice/crio-b3d5b2cff1494cae88ba04350c4d5e6726462ed12caaced1883f09d3ca7aac5c WatchSource:0}: Error finding container b3d5b2cff1494cae88ba04350c4d5e6726462ed12caaced1883f09d3ca7aac5c: Status 404 returned error can't find the container with id b3d5b2cff1494cae88ba04350c4d5e6726462ed12caaced1883f09d3ca7aac5c Jul 22 16:41:22.709944 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.709918 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8"] Jul 22 16:41:22.712842 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.712818 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86675ff59-gknp5"] Jul 22 16:41:22.716402 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.715567 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-55cc957bbb-8brdb"] Jul 22 16:41:22.716656 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.716548 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x"] Jul 22 16:41:22.953720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:22.953641 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hxzdx" event={"ID":"2463af8f-0af9-4ff6-ba3b-d944175ebca1","Type":"ContainerStarted","Data":"b3d5b2cff1494cae88ba04350c4d5e6726462ed12caaced1883f09d3ca7aac5c"} Jul 22 16:41:24.314630 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:24.314591 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b9cee3b_a61a_4de1_9b7d_21d1f68b6b2a.slice/crio-7e20cfd566f843028c29e670460dee64b3cd1e4da10598cf3343b8822a635fa6 WatchSource:0}: Error finding container 7e20cfd566f843028c29e670460dee64b3cd1e4da10598cf3343b8822a635fa6: Status 404 returned error can't find the container with id 7e20cfd566f843028c29e670460dee64b3cd1e4da10598cf3343b8822a635fa6 Jul 22 16:41:24.316566 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:24.316540 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88cc0d4_8c3d_4c35_9326_ce054e6cf331.slice/crio-9213b731509b7c211de428b53d01180745bff279d8fea25047b09605f1b1bcfa WatchSource:0}: Error finding container 9213b731509b7c211de428b53d01180745bff279d8fea25047b09605f1b1bcfa: Status 404 returned error can't find the container with id 9213b731509b7c211de428b53d01180745bff279d8fea25047b09605f1b1bcfa Jul 22 16:41:24.445824 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:24.445796 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c3c642_652a_4b5f_b150_04e8b0c7b074.slice/crio-abfa896c49ec318d21696c460bdb728b0a13d9a9bc21f656f1ae917ee7229b44 WatchSource:0}: Error finding container abfa896c49ec318d21696c460bdb728b0a13d9a9bc21f656f1ae917ee7229b44: Status 404 returned error can't find the container with id abfa896c49ec318d21696c460bdb728b0a13d9a9bc21f656f1ae917ee7229b44 Jul 22 16:41:24.961769 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:24.961731 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" event={"ID":"a3c3c642-652a-4b5f-b150-04e8b0c7b074","Type":"ContainerStarted","Data":"abfa896c49ec318d21696c460bdb728b0a13d9a9bc21f656f1ae917ee7229b44"} Jul 22 16:41:24.963071 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:24.963039 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" event={"ID":"cf0d3172-937e-483a-b1c0-da9606f674a9","Type":"ContainerStarted","Data":"47ff365f00413edf3622117031201b8ac3306c38e27e76eb03ad53744c466af8"} Jul 22 16:41:24.964337 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:24.964299 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-55cc957bbb-8brdb" event={"ID":"f88cc0d4-8c3d-4c35-9326-ce054e6cf331","Type":"ContainerStarted","Data":"9213b731509b7c211de428b53d01180745bff279d8fea25047b09605f1b1bcfa"} Jul 22 16:41:24.965749 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:24.965709 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86675ff59-gknp5" event={"ID":"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a","Type":"ContainerStarted","Data":"7e20cfd566f843028c29e670460dee64b3cd1e4da10598cf3343b8822a635fa6"} Jul 22 16:41:27.531499 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.529549 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb"] Jul 22 16:41:27.538161 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.538133 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" Jul 22 16:41:27.541348 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.540934 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 16:41:27.541348 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.541250 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 16:41:27.542198 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.542023 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zwwj2\"" Jul 22 16:41:27.553796 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.553763 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb"] Jul 22 16:41:27.654025 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.653980 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/60ea9185-8846-4c18-916a-8587d9c0877a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-f2pjb\" (UID: \"60ea9185-8846-4c18-916a-8587d9c0877a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" Jul 22 16:41:27.654201 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.654108 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/60ea9185-8846-4c18-916a-8587d9c0877a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-f2pjb\" (UID: \"60ea9185-8846-4c18-916a-8587d9c0877a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" Jul 22 16:41:27.755244 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.755198 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/60ea9185-8846-4c18-916a-8587d9c0877a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-f2pjb\" (UID: \"60ea9185-8846-4c18-916a-8587d9c0877a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" Jul 22 16:41:27.755431 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.755266 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/60ea9185-8846-4c18-916a-8587d9c0877a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-f2pjb\" (UID: \"60ea9185-8846-4c18-916a-8587d9c0877a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" Jul 22 16:41:27.755879 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.755818 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/60ea9185-8846-4c18-916a-8587d9c0877a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-f2pjb\" (UID: \"60ea9185-8846-4c18-916a-8587d9c0877a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" Jul 22 16:41:27.758283 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.758232 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/60ea9185-8846-4c18-916a-8587d9c0877a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-f2pjb\" (UID: \"60ea9185-8846-4c18-916a-8587d9c0877a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" Jul 22 16:41:27.852893 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.852821 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" Jul 22 16:41:27.977608 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.977575 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ld4x4" event={"ID":"eaf250c9-d810-4a19-9844-b0e5fa5b8526","Type":"ContainerStarted","Data":"b95f6d174b38822194db93374ba28001ffa8bc0fc3dfc5e1a6d9f06d903f04b8"} Jul 22 16:41:27.978888 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.978858 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dmtxl" event={"ID":"e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a","Type":"ContainerStarted","Data":"8e0fd45e8afcc2267691bb2cf5e0ac96945cccdd734756b87d2468a05cf5213b"} Jul 22 16:41:27.980245 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.980220 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-55cc957bbb-8brdb" event={"ID":"f88cc0d4-8c3d-4c35-9326-ce054e6cf331","Type":"ContainerStarted","Data":"f09be142e838c13589e6654648fc8ecddaba10551fbe7b1be7a5b142c5838cb2"} Jul 22 16:41:27.981571 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.981548 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86675ff59-gknp5" event={"ID":"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a","Type":"ContainerStarted","Data":"ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5"} Jul 22 16:41:27.981727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.981680 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:41:27.997293 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:27.997228 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dmtxl" podStartSLOduration=67.891472323 podStartE2EDuration="1m13.99721047s" totalImagesPullingTime="6.105738147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:27.996633664 +0000 UTC m=+74.907290842" watchObservedRunningTime="2026-07-22 16:41:27.99721047 +0000 UTC m=+74.907867643" Jul 22 16:41:28.044117 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:28.044073 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-55cc957bbb-8brdb" podStartSLOduration=63.044058634 podStartE2EDuration="1m3.044058634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:28.043350557 +0000 UTC m=+74.954007730" watchObservedRunningTime="2026-07-22 16:41:28.044058634 +0000 UTC m=+74.954715801" Jul 22 16:41:28.044708 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:28.044667 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86675ff59-gknp5" podStartSLOduration=74.044652633 podStartE2EDuration="1m14.044652633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:28.021329707 +0000 UTC m=+74.931986880" watchObservedRunningTime="2026-07-22 16:41:28.044652633 +0000 UTC m=+74.955309804" Jul 22 16:41:28.110326 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:28.110255 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:28.112834 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:28.112808 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:28.985432 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:28.985402 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:28.986914 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:28.986891 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-55cc957bbb-8brdb" Jul 22 16:41:29.921762 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:29.921694 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb"] Jul 22 16:41:29.926306 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:29.926262 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ea9185_8846_4c18_916a_8587d9c0877a.slice/crio-14f2591af994440b67274e3c871bf58edc62a897037d90a5bdc457406da08c21 WatchSource:0}: Error finding container 14f2591af994440b67274e3c871bf58edc62a897037d90a5bdc457406da08c21: Status 404 returned error can't find the container with id 14f2591af994440b67274e3c871bf58edc62a897037d90a5bdc457406da08c21 Jul 22 16:41:29.994637 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:29.994567 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hxzdx" event={"ID":"2463af8f-0af9-4ff6-ba3b-d944175ebca1","Type":"ContainerStarted","Data":"cbf38cd6e20a521f751346c37a5768fe1e2113cabd5691604f0ef5946738e61a"} Jul 22 16:41:29.997851 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:29.997685 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" event={"ID":"a3c3c642-652a-4b5f-b150-04e8b0c7b074","Type":"ContainerStarted","Data":"acdfd422dc228ff858ae2220e568868e3877e421c407986f9b9e80e9fd2024f8"} Jul 22 16:41:29.999407 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:29.999355 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" event={"ID":"cf0d3172-937e-483a-b1c0-da9606f674a9","Type":"ContainerStarted","Data":"54baa39a36b17549d920f9ba05341d8e49b6974ddff5e6e32c8a0ac6a62b5778"} Jul 22 16:41:30.000896 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.000871 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" event={"ID":"60ea9185-8846-4c18-916a-8587d9c0877a","Type":"ContainerStarted","Data":"14f2591af994440b67274e3c871bf58edc62a897037d90a5bdc457406da08c21"} Jul 22 16:41:30.002833 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.002809 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ld4x4" event={"ID":"eaf250c9-d810-4a19-9844-b0e5fa5b8526","Type":"ContainerStarted","Data":"e83bcd3f4258317ec084add0ce2b48531d937ec8b0893e7432eab89c62c8856f"} Jul 22 16:41:30.005389 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.005344 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfzrg" event={"ID":"a1409262-765e-45c5-97b1-df28ad5043ed","Type":"ContainerStarted","Data":"69f100b03f2b108918fe6aed29581a0c30c31701316d0c255147a080a74c0e23"} Jul 22 16:41:30.041651 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.041604 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ld4x4" podStartSLOduration=70.261657724 podStartE2EDuration="1m16.041586193s" totalImagesPullingTime="5.779928469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:30.041276249 +0000 UTC m=+76.951933422" watchObservedRunningTime="2026-07-22 16:41:30.041586193 +0000 UTC m=+76.952243369" Jul 22 16:41:30.119261 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.119202 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-h8th8" podStartSLOduration=59.792738435 podStartE2EDuration="1m5.11918158s" totalImagesPullingTime="5.326443145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:30.117262903 +0000 UTC m=+77.027920078" watchObservedRunningTime="2026-07-22 16:41:30.11918158 +0000 UTC m=+77.029838752" Jul 22 16:41:30.481789 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.481751 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb"] Jul 22 16:41:30.484635 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.484611 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb" Jul 22 16:41:30.486974 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.486949 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 22 16:41:30.487223 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.487198 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-btrp8\"" Jul 22 16:41:30.494029 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.493968 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb"] Jul 22 16:41:30.578970 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.578937 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e2da78f6-f615-4ded-b42f-773e4aef5229-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-xlscb\" (UID: \"e2da78f6-f615-4ded-b42f-773e4aef5229\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb" Jul 22 16:41:30.679639 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.679593 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e2da78f6-f615-4ded-b42f-773e4aef5229-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-xlscb\" (UID: \"e2da78f6-f615-4ded-b42f-773e4aef5229\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb" Jul 22 16:41:30.682873 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.682823 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e2da78f6-f615-4ded-b42f-773e4aef5229-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-xlscb\" (UID: \"e2da78f6-f615-4ded-b42f-773e4aef5229\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb" Jul 22 16:41:30.796290 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.796254 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb" Jul 22 16:41:30.957740 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:30.957502 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb"] Jul 22 16:41:31.012772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:31.012732 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hxzdx" event={"ID":"2463af8f-0af9-4ff6-ba3b-d944175ebca1","Type":"ContainerStarted","Data":"c13b121c922c0f71dc642103191445838733d6af1cf7cc905bab454416a4a809"} Jul 22 16:41:31.013218 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:31.012877 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hxzdx" Jul 22 16:41:31.015429 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:31.015396 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" event={"ID":"cf0d3172-937e-483a-b1c0-da9606f674a9","Type":"ContainerStarted","Data":"72a15216e9cb87a4601df84eb014258c7b8e4bf3cab2ff331cf4ca0a69efe75c"} Jul 22 16:41:31.033523 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:31.033469 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hxzdx" podStartSLOduration=34.950390225 podStartE2EDuration="42.033449423s" totalImagesPullingTime="7.083059198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:31.03188951 +0000 UTC m=+77.942546695" watchObservedRunningTime="2026-07-22 16:41:31.033449423 +0000 UTC m=+77.944106596" Jul 22 16:41:31.049696 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:31.049601 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-8659x" podStartSLOduration=50.648623434 podStartE2EDuration="56.049584625s" totalImagesPullingTime="5.400961191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:31.048057682 +0000 UTC m=+77.958714871" watchObservedRunningTime="2026-07-22 16:41:31.049584625 +0000 UTC m=+77.960241800" Jul 22 16:41:31.212033 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:31.211994 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2da78f6_f615_4ded_b42f_773e4aef5229.slice/crio-bdf99c4b86b0e110aba86942078ebdea3d0c3b78d60918fc19cc09fbb773608b WatchSource:0}: Error finding container bdf99c4b86b0e110aba86942078ebdea3d0c3b78d60918fc19cc09fbb773608b: Status 404 returned error can't find the container with id bdf99c4b86b0e110aba86942078ebdea3d0c3b78d60918fc19cc09fbb773608b Jul 22 16:41:32.021419 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:32.021356 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb" event={"ID":"e2da78f6-f615-4ded-b42f-773e4aef5229","Type":"ContainerStarted","Data":"bdf99c4b86b0e110aba86942078ebdea3d0c3b78d60918fc19cc09fbb773608b"} Jul 22 16:41:32.023634 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:32.023605 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" event={"ID":"60ea9185-8846-4c18-916a-8587d9c0877a","Type":"ContainerStarted","Data":"3c55ed39f6462003d2fb2c11c1605dcb6ce633cc9e3a24f63c86e9357206aa86"} Jul 22 16:41:32.041407 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:32.041332 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-f2pjb" podStartSLOduration=3.709760755 podStartE2EDuration="5.04131298s" totalImagesPullingTime="1.331552225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:32.040159811 +0000 UTC m=+78.950816984" watchObservedRunningTime="2026-07-22 16:41:32.04131298 +0000 UTC m=+78.951970153" Jul 22 16:41:33.614496 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:33.614466 2581 scope.go:117] "RemoveContainer" containerID="8ebe7c0964cbbd7470bd9d2b3df643b4fe351509688fcb1483b9011ec1c47298" Jul 22 16:41:33.614938 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:33.614640 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-m5hgc_openshift-console-operator(3bde7f11-4339-4d78-8531-b12f8d734aec)\"" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" podUID="3bde7f11-4339-4d78-8531-b12f8d734aec" Jul 22 16:41:34.038724 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:34.038676 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb" event={"ID":"e2da78f6-f615-4ded-b42f-773e4aef5229","Type":"ContainerStarted","Data":"56a4160529dda226dcbc47ae971fcc6c5c869c41b944c23c00113b1ea4cf2415"} Jul 22 16:41:34.038898 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:34.038770 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb" Jul 22 16:41:34.040760 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:34.040728 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfzrg" event={"ID":"a1409262-765e-45c5-97b1-df28ad5043ed","Type":"ContainerStarted","Data":"d8174d765eae1095fc0ea65a0d9ae675d1af532c7141c6d9b7c5daab6e373ba5"} Jul 22 16:41:34.043637 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:34.043606 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb" Jul 22 16:41:34.055334 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:34.055295 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-xlscb" podStartSLOduration=1.456432972 podStartE2EDuration="4.055280024s" totalImagesPullingTime="2.598847052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:34.054717173 +0000 UTC m=+80.965374323" watchObservedRunningTime="2026-07-22 16:41:34.055280024 +0000 UTC m=+80.965937201" Jul 22 16:41:34.073740 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:34.073704 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gfzrg" podStartSLOduration=17.335005447 podStartE2EDuration="35.073695995s" totalImagesPullingTime="17.738690548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:40:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:34.072609966 +0000 UTC m=+80.983267138" watchObservedRunningTime="2026-07-22 16:41:34.073695995 +0000 UTC m=+80.984353181" Jul 22 16:41:41.027990 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:41.027563 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hxzdx" Jul 22 16:41:42.104179 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.104146 2581 patch_prober.go:28] interesting pod/image-registry-86675ff59-gknp5 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 22 16:41:42.104624 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.104198 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-86675ff59-gknp5" podUID="9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 16:41:42.902242 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.902208 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ql5bm"] Jul 22 16:41:42.907061 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.907041 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:42.910042 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.910010 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 16:41:42.910137 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.910016 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 16:41:42.910137 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.910094 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 16:41:42.910407 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.910386 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qv2jv\"" Jul 22 16:41:42.910899 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.910882 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 16:41:42.965036 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.965010 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-accelerators-collector-config\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:42.965121 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.965042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64f3f420-7bb6-45e9-832e-3601a5ee6b16-root\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:42.965121 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.965063 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64f3f420-7bb6-45e9-832e-3601a5ee6b16-sys\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:42.965121 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.965078 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:42.965220 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.965126 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-wtmp\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:42.965220 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.965169 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-tls\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:42.965220 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.965197 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-textfile\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:42.965309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.965215 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpjw9\" (UniqueName: \"kubernetes.io/projected/64f3f420-7bb6-45e9-832e-3601a5ee6b16-kube-api-access-zpjw9\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:42.965309 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:42.965242 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64f3f420-7bb6-45e9-832e-3601a5ee6b16-metrics-client-ca\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.066471 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066442 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-wtmp\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.066609 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066488 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-tls\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.066609 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066515 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-textfile\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.066609 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpjw9\" (UniqueName: \"kubernetes.io/projected/64f3f420-7bb6-45e9-832e-3601a5ee6b16-kube-api-access-zpjw9\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.066776 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066608 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-wtmp\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.066776 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066622 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64f3f420-7bb6-45e9-832e-3601a5ee6b16-metrics-client-ca\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.066776 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:43.066685 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 22 16:41:43.066776 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066689 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-accelerators-collector-config\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.066776 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:41:43.066762 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-tls podName:64f3f420-7bb6-45e9-832e-3601a5ee6b16 nodeName:}" failed. No retries permitted until 2026-07-22 16:41:43.566730813 +0000 UTC m=+90.477387972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-tls") pod "node-exporter-ql5bm" (UID: "64f3f420-7bb6-45e9-832e-3601a5ee6b16") : secret "node-exporter-tls" not found Jul 22 16:41:43.067005 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066823 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-textfile\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.067005 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64f3f420-7bb6-45e9-832e-3601a5ee6b16-root\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.067005 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066875 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64f3f420-7bb6-45e9-832e-3601a5ee6b16-sys\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.067005 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066881 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64f3f420-7bb6-45e9-832e-3601a5ee6b16-root\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.067005 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066900 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.067005 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.066924 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64f3f420-7bb6-45e9-832e-3601a5ee6b16-sys\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.067189 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.067138 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64f3f420-7bb6-45e9-832e-3601a5ee6b16-metrics-client-ca\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.067189 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.067137 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-accelerators-collector-config\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.069029 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.069006 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.077972 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.077949 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpjw9\" (UniqueName: \"kubernetes.io/projected/64f3f420-7bb6-45e9-832e-3601a5ee6b16-kube-api-access-zpjw9\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.571657 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.571619 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-tls\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.573841 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.573821 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64f3f420-7bb6-45e9-832e-3601a5ee6b16-node-exporter-tls\") pod \"node-exporter-ql5bm\" (UID: \"64f3f420-7bb6-45e9-832e-3601a5ee6b16\") " pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.816768 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:43.816738 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ql5bm" Jul 22 16:41:43.830506 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:43.830484 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f3f420_7bb6_45e9_832e_3601a5ee6b16.slice/crio-160bc91d56b443d15054b9eb94c2de1a190fa1f3b69d2ebb7e19e60ff637f1bd WatchSource:0}: Error finding container 160bc91d56b443d15054b9eb94c2de1a190fa1f3b69d2ebb7e19e60ff637f1bd: Status 404 returned error can't find the container with id 160bc91d56b443d15054b9eb94c2de1a190fa1f3b69d2ebb7e19e60ff637f1bd Jul 22 16:41:44.075770 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:44.075728 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ql5bm" event={"ID":"64f3f420-7bb6-45e9-832e-3601a5ee6b16","Type":"ContainerStarted","Data":"160bc91d56b443d15054b9eb94c2de1a190fa1f3b69d2ebb7e19e60ff637f1bd"} Jul 22 16:41:45.081448 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:45.081417 2581 generic.go:356] "Generic (PLEG): container finished" podID="64f3f420-7bb6-45e9-832e-3601a5ee6b16" containerID="cebec49974ad2ea12b934846ba474b4c75dd339fb7b9a50cbdd738bd7d156203" exitCode=0 Jul 22 16:41:45.081778 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:45.081496 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ql5bm" event={"ID":"64f3f420-7bb6-45e9-832e-3601a5ee6b16","Type":"ContainerDied","Data":"cebec49974ad2ea12b934846ba474b4c75dd339fb7b9a50cbdd738bd7d156203"} Jul 22 16:41:46.087078 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:46.087045 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ql5bm" event={"ID":"64f3f420-7bb6-45e9-832e-3601a5ee6b16","Type":"ContainerStarted","Data":"608893feeb4171aa3137d36c9e8aa724aa160294e8d5e831daf815c4d0c66be8"} Jul 22 16:41:46.087463 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:46.087096 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ql5bm" event={"ID":"64f3f420-7bb6-45e9-832e-3601a5ee6b16","Type":"ContainerStarted","Data":"b89e46df5e3dc6a09fb66e6bd08fe6f42dc1fc94e1ac8e16b012c51b62ca9bcd"} Jul 22 16:41:46.111562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:46.111518 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ql5bm" podStartSLOduration=3.167029721 podStartE2EDuration="4.111502362s" totalImagesPullingTime="882.620699ms" totalInitContainerRuntime="61.851942ms" isStatefulPod=true podCreationTimestamp="2026-07-22 16:41:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:46.111125644 +0000 UTC m=+93.021782817" watchObservedRunningTime="2026-07-22 16:41:46.111502362 +0000 UTC m=+93.022159534" Jul 22 16:41:47.617605 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:47.617579 2581 scope.go:117] "RemoveContainer" containerID="8ebe7c0964cbbd7470bd9d2b3df643b4fe351509688fcb1483b9011ec1c47298" Jul 22 16:41:48.095625 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:48.095599 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 16:41:48.095771 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:48.095686 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" event={"ID":"3bde7f11-4339-4d78-8531-b12f8d734aec","Type":"ContainerStarted","Data":"71d596992d7eeefaaefb2679fe88c1c70b12d7824a9dc1e40312842f63a9d23b"} Jul 22 16:41:48.096002 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:48.095972 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:41:48.128085 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:48.128008 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" podStartSLOduration=65.550360491 podStartE2EDuration="1m13.127989465s" totalImagesPullingTime="7.577628974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:48.126788542 +0000 UTC m=+95.037445714" watchObservedRunningTime="2026-07-22 16:41:48.127989465 +0000 UTC m=+95.038646644" Jul 22 16:41:48.248600 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:48.248575 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-m5hgc" Jul 22 16:41:48.990019 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:48.989994 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:41:51.950625 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:51.950594 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jfkr8" Jul 22 16:41:53.600087 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:53.600056 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86675ff59-gknp5"] Jul 22 16:41:57.363587 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.363549 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cd7b4788c-g2l9f"] Jul 22 16:41:57.487563 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.487532 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd7b4788c-g2l9f"] Jul 22 16:41:57.487713 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.487630 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.490154 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.490131 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 16:41:57.490331 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.490180 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 16:41:57.490331 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.490240 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 16:41:57.490511 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.490495 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 16:41:57.490585 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.490515 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 16:41:57.490585 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.490554 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pktll\"" Jul 22 16:41:57.491140 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.491122 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 16:41:57.491230 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.491177 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 16:41:57.566773 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.566744 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xnwx\" (UniqueName: \"kubernetes.io/projected/1ce2cc23-bf63-4325-b06e-8b07d503ed49-kube-api-access-4xnwx\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.566875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.566787 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-serving-cert\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.566875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.566859 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-service-ca\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.566947 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.566883 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-oauth-serving-cert\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.566947 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.566902 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-config\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.566947 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.566923 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-oauth-config\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.668002 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.667952 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-oauth-config\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.668002 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.667993 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xnwx\" (UniqueName: \"kubernetes.io/projected/1ce2cc23-bf63-4325-b06e-8b07d503ed49-kube-api-access-4xnwx\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.668165 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.668010 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-serving-cert\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.668165 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.668136 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-service-ca\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.668252 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.668166 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-oauth-serving-cert\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.668252 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.668190 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-config\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.668811 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.668791 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-config\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.668811 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.668800 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-oauth-serving-cert\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.668968 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.668848 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-service-ca\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.670395 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.670360 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-serving-cert\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.670814 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.670796 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-oauth-config\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.675250 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.675230 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xnwx\" (UniqueName: \"kubernetes.io/projected/1ce2cc23-bf63-4325-b06e-8b07d503ed49-kube-api-access-4xnwx\") pod \"console-cd7b4788c-g2l9f\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.796943 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.796920 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:41:57.967673 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:57.967648 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd7b4788c-g2l9f"] Jul 22 16:41:57.970045 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:41:57.970018 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ce2cc23_bf63_4325_b06e_8b07d503ed49.slice/crio-0d0bb572ab77a06cb9da37917c3fba87e65abb5acd7ba956351fc4834590bd56 WatchSource:0}: Error finding container 0d0bb572ab77a06cb9da37917c3fba87e65abb5acd7ba956351fc4834590bd56: Status 404 returned error can't find the container with id 0d0bb572ab77a06cb9da37917c3fba87e65abb5acd7ba956351fc4834590bd56 Jul 22 16:41:58.133144 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:41:58.133113 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd7b4788c-g2l9f" event={"ID":"1ce2cc23-bf63-4325-b06e-8b07d503ed49","Type":"ContainerStarted","Data":"0d0bb572ab77a06cb9da37917c3fba87e65abb5acd7ba956351fc4834590bd56"} Jul 22 16:42:02.152864 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:02.152827 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd7b4788c-g2l9f" event={"ID":"1ce2cc23-bf63-4325-b06e-8b07d503ed49","Type":"ContainerStarted","Data":"a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125"} Jul 22 16:42:02.172743 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:02.172694 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cd7b4788c-g2l9f" podStartSLOduration=1.530461344 podStartE2EDuration="5.172679873s" totalImagesPullingTime="3.642218529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:42:02.171096425 +0000 UTC m=+109.081753596" watchObservedRunningTime="2026-07-22 16:42:02.172679873 +0000 UTC m=+109.083337044" Jul 22 16:42:05.534275 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.534241 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bbd447985-9xb5h"] Jul 22 16:42:05.537270 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.537249 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.544263 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.544243 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 16:42:05.550306 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.550281 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bbd447985-9xb5h"] Jul 22 16:42:05.628935 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.628911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-service-ca\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.629023 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.628944 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfnxs\" (UniqueName: \"kubernetes.io/projected/790a6b95-011b-428e-9910-9fa99a0c19df-kube-api-access-hfnxs\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.629023 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.628976 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-trusted-ca-bundle\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.629098 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.629062 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-serving-cert\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.629098 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.629090 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-console-config\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.629161 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.629121 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-oauth-config\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.629161 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.629148 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-oauth-serving-cert\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.729784 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.729756 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-oauth-serving-cert\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.729911 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.729796 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-service-ca\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.729911 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.729814 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfnxs\" (UniqueName: \"kubernetes.io/projected/790a6b95-011b-428e-9910-9fa99a0c19df-kube-api-access-hfnxs\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.729911 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.729852 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-trusted-ca-bundle\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.730038 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.729911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-serving-cert\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.730195 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.730172 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-console-config\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.730262 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.730231 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-oauth-config\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.730611 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.730588 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-service-ca\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.730717 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.730593 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-oauth-serving-cert\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.730852 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.730828 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-trusted-ca-bundle\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.730970 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.730951 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-console-config\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.732580 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.732555 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-serving-cert\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.732703 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.732561 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-oauth-config\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.743678 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.743658 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfnxs\" (UniqueName: \"kubernetes.io/projected/790a6b95-011b-428e-9910-9fa99a0c19df-kube-api-access-hfnxs\") pod \"console-5bbd447985-9xb5h\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.846330 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.846281 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:05.970384 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:05.970257 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bbd447985-9xb5h"] Jul 22 16:42:05.972950 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:42:05.972924 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod790a6b95_011b_428e_9910_9fa99a0c19df.slice/crio-10ebeafb8824b028b471fa75ebb5bfd8b3c35c3c2ab1e5f2ea0691f1b30fbe59 WatchSource:0}: Error finding container 10ebeafb8824b028b471fa75ebb5bfd8b3c35c3c2ab1e5f2ea0691f1b30fbe59: Status 404 returned error can't find the container with id 10ebeafb8824b028b471fa75ebb5bfd8b3c35c3c2ab1e5f2ea0691f1b30fbe59 Jul 22 16:42:06.168840 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:06.168753 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bbd447985-9xb5h" event={"ID":"790a6b95-011b-428e-9910-9fa99a0c19df","Type":"ContainerStarted","Data":"a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73"} Jul 22 16:42:06.168840 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:06.168806 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bbd447985-9xb5h" event={"ID":"790a6b95-011b-428e-9910-9fa99a0c19df","Type":"ContainerStarted","Data":"10ebeafb8824b028b471fa75ebb5bfd8b3c35c3c2ab1e5f2ea0691f1b30fbe59"} Jul 22 16:42:06.200824 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:06.200772 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bbd447985-9xb5h" podStartSLOduration=1.200757631 podStartE2EDuration="1.200757631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:42:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:42:06.200607088 +0000 UTC m=+113.111264257" watchObservedRunningTime="2026-07-22 16:42:06.200757631 +0000 UTC m=+113.111414803" Jul 22 16:42:07.797919 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:07.797880 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:42:07.798268 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:07.797941 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:42:07.802093 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:07.802070 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:42:08.180292 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:08.180226 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:42:10.184741 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:10.184705 2581 generic.go:356] "Generic (PLEG): container finished" podID="fa257d23-f7ea-498f-a500-b653baad12c8" containerID="62ded8fb6509578eb5fa1e70d821b0eec5547c10f6764d791643de0c13debb20" exitCode=0 Jul 22 16:42:10.185090 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:10.184747 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lv47m" event={"ID":"fa257d23-f7ea-498f-a500-b653baad12c8","Type":"ContainerDied","Data":"62ded8fb6509578eb5fa1e70d821b0eec5547c10f6764d791643de0c13debb20"} Jul 22 16:42:10.185090 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:10.185034 2581 scope.go:117] "RemoveContainer" containerID="62ded8fb6509578eb5fa1e70d821b0eec5547c10f6764d791643de0c13debb20" Jul 22 16:42:11.190346 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:11.190314 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lv47m" event={"ID":"fa257d23-f7ea-498f-a500-b653baad12c8","Type":"ContainerStarted","Data":"b222491e343d61d65299ce3c506bdbd2bd2321059a93378ed233f9abe0a34291"} Jul 22 16:42:14.202780 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:14.202708 2581 generic.go:356] "Generic (PLEG): container finished" podID="dec7161d-014c-45dc-a3ac-9b61d0f1be80" containerID="5921720096b12946b5100c84882968d2533a7fdab20fc4681e8d043d0bbe9458" exitCode=0 Jul 22 16:42:14.203110 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:14.202779 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" event={"ID":"dec7161d-014c-45dc-a3ac-9b61d0f1be80","Type":"ContainerDied","Data":"5921720096b12946b5100c84882968d2533a7fdab20fc4681e8d043d0bbe9458"} Jul 22 16:42:14.203110 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:14.203068 2581 scope.go:117] "RemoveContainer" containerID="5921720096b12946b5100c84882968d2533a7fdab20fc4681e8d043d0bbe9458" Jul 22 16:42:15.208706 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:15.208669 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9jmsx" event={"ID":"dec7161d-014c-45dc-a3ac-9b61d0f1be80","Type":"ContainerStarted","Data":"c62b13eb372f4d2768808bb6c9a7efc1691a759d347a1ab450b9e8e747318f79"} Jul 22 16:42:15.846654 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:15.846603 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:15.846654 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:15.846673 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:15.851495 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:15.851474 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:16.216347 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:16.216277 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:42:16.259306 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:16.259271 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd7b4788c-g2l9f"] Jul 22 16:42:18.622473 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:18.622405 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86675ff59-gknp5" podUID="9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" containerName="registry" containerID="cri-o://ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5" gracePeriod=30 Jul 22 16:42:18.888222 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:18.888161 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:42:19.029834 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.029804 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-certificates\") pod \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " Jul 22 16:42:19.029934 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.029850 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlmq2\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-kube-api-access-xlmq2\") pod \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " Jul 22 16:42:19.029934 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.029886 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-ca-trust-extracted\") pod \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " Jul 22 16:42:19.029934 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.029916 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-image-registry-private-configuration\") pod \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " Jul 22 16:42:19.030084 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.029948 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") pod \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " Jul 22 16:42:19.030084 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.029986 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-bound-sa-token\") pod \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " Jul 22 16:42:19.030184 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.030103 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-trusted-ca\") pod \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " Jul 22 16:42:19.030243 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.030183 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:42:19.030243 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.030192 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-installation-pull-secrets\") pod \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\" (UID: \"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a\") " Jul 22 16:42:19.030680 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.030456 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-certificates\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:19.030680 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.030503 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:42:19.032340 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.032298 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:42:19.032571 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.032547 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:42:19.032648 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.032607 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:42:19.032742 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.032723 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-kube-api-access-xlmq2" (OuterVolumeSpecName: "kube-api-access-xlmq2") pod "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a"). InnerVolumeSpecName "kube-api-access-xlmq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:42:19.032790 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.032730 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:42:19.038565 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.038543 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" (UID: "9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:42:19.131781 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.131755 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-registry-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:19.131781 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.131777 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-bound-sa-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:19.131781 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.131785 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-trusted-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:19.131937 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.131793 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-installation-pull-secrets\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:19.131937 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.131802 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlmq2\" (UniqueName: \"kubernetes.io/projected/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-kube-api-access-xlmq2\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:19.131937 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.131809 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-ca-trust-extracted\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:19.131937 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.131819 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a-image-registry-private-configuration\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:19.225165 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.225138 2581 generic.go:356] "Generic (PLEG): container finished" podID="9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" containerID="ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5" exitCode=0 Jul 22 16:42:19.225269 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.225182 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86675ff59-gknp5" Jul 22 16:42:19.225269 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.225228 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86675ff59-gknp5" event={"ID":"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a","Type":"ContainerDied","Data":"ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5"} Jul 22 16:42:19.225403 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.225324 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86675ff59-gknp5" event={"ID":"9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a","Type":"ContainerDied","Data":"7e20cfd566f843028c29e670460dee64b3cd1e4da10598cf3343b8822a635fa6"} Jul 22 16:42:19.225403 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.225357 2581 scope.go:117] "RemoveContainer" containerID="ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5" Jul 22 16:42:19.226813 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.226785 2581 generic.go:356] "Generic (PLEG): container finished" podID="7967225e-9262-443f-b76a-e532304e06f9" containerID="82c81b4ccf77a1b47d90da1960c78e058a908d8fb78889f24ed411a048e6581d" exitCode=0 Jul 22 16:42:19.226894 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.226856 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" event={"ID":"7967225e-9262-443f-b76a-e532304e06f9","Type":"ContainerDied","Data":"82c81b4ccf77a1b47d90da1960c78e058a908d8fb78889f24ed411a048e6581d"} Jul 22 16:42:19.227239 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.227221 2581 scope.go:117] "RemoveContainer" containerID="82c81b4ccf77a1b47d90da1960c78e058a908d8fb78889f24ed411a048e6581d" Jul 22 16:42:19.236471 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.236450 2581 scope.go:117] "RemoveContainer" containerID="ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5" Jul 22 16:42:19.236720 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:42:19.236702 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5\": container with ID starting with ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5 not found: ID does not exist" containerID="ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5" Jul 22 16:42:19.236796 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.236732 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5"} err="failed to get container status \"ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5\": rpc error: code = NotFound desc = could not find container \"ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5\": container with ID starting with ae2a6c4bab3a8414d5228ee9c6db15f760fc7c8a5d9d5a18f0e6b108609fadb5 not found: ID does not exist" Jul 22 16:42:19.259681 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.259655 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86675ff59-gknp5"] Jul 22 16:42:19.267164 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.267142 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86675ff59-gknp5"] Jul 22 16:42:19.618403 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:19.618314 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" path="/var/lib/kubelet/pods/9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a/volumes" Jul 22 16:42:20.234437 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:20.233473 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8mgm7" event={"ID":"7967225e-9262-443f-b76a-e532304e06f9","Type":"ContainerStarted","Data":"d2c61e672746038750c678fa1d6baf355b22f30df4f1c7f749e6521f0034f226"} Jul 22 16:42:41.281474 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.281431 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cd7b4788c-g2l9f" podUID="1ce2cc23-bf63-4325-b06e-8b07d503ed49" containerName="console" containerID="cri-o://a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125" gracePeriod=15 Jul 22 16:42:41.528172 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.528146 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:42:41.696875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.696816 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-service-ca\") pod \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " Jul 22 16:42:41.696875 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.696850 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-serving-cert\") pod \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " Jul 22 16:42:41.697021 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.696880 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xnwx\" (UniqueName: \"kubernetes.io/projected/1ce2cc23-bf63-4325-b06e-8b07d503ed49-kube-api-access-4xnwx\") pod \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " Jul 22 16:42:41.697073 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.697055 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-config\") pod \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " Jul 22 16:42:41.697120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.697092 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-oauth-serving-cert\") pod \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " Jul 22 16:42:41.697172 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.697142 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-oauth-config\") pod \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\" (UID: \"1ce2cc23-bf63-4325-b06e-8b07d503ed49\") " Jul 22 16:42:41.697249 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.697219 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-service-ca" (OuterVolumeSpecName: "service-ca") pod "1ce2cc23-bf63-4325-b06e-8b07d503ed49" (UID: "1ce2cc23-bf63-4325-b06e-8b07d503ed49"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:42:41.697733 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.697451 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-config" (OuterVolumeSpecName: "console-config") pod "1ce2cc23-bf63-4325-b06e-8b07d503ed49" (UID: "1ce2cc23-bf63-4325-b06e-8b07d503ed49"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:42:41.697733 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.697491 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1ce2cc23-bf63-4325-b06e-8b07d503ed49" (UID: "1ce2cc23-bf63-4325-b06e-8b07d503ed49"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:42:41.697733 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.697578 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:41.697733 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.697595 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-oauth-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:41.697733 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.697609 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce2cc23-bf63-4325-b06e-8b07d503ed49-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:41.699140 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.699109 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1ce2cc23-bf63-4325-b06e-8b07d503ed49" (UID: "1ce2cc23-bf63-4325-b06e-8b07d503ed49"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:42:41.699238 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.699147 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1ce2cc23-bf63-4325-b06e-8b07d503ed49" (UID: "1ce2cc23-bf63-4325-b06e-8b07d503ed49"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:42:41.699238 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.699196 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ce2cc23-bf63-4325-b06e-8b07d503ed49-kube-api-access-4xnwx" (OuterVolumeSpecName: "kube-api-access-4xnwx") pod "1ce2cc23-bf63-4325-b06e-8b07d503ed49" (UID: "1ce2cc23-bf63-4325-b06e-8b07d503ed49"). InnerVolumeSpecName "kube-api-access-4xnwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:42:41.798542 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.798517 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-oauth-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:41.798542 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.798541 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce2cc23-bf63-4325-b06e-8b07d503ed49-console-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:41.798675 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:41.798555 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xnwx\" (UniqueName: \"kubernetes.io/projected/1ce2cc23-bf63-4325-b06e-8b07d503ed49-kube-api-access-4xnwx\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:42:42.322247 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:42.322213 2581 generic.go:356] "Generic (PLEG): container finished" podID="1ce2cc23-bf63-4325-b06e-8b07d503ed49" containerID="a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125" exitCode=0 Jul 22 16:42:42.322633 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:42.322274 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd7b4788c-g2l9f" Jul 22 16:42:42.322633 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:42.322297 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd7b4788c-g2l9f" event={"ID":"1ce2cc23-bf63-4325-b06e-8b07d503ed49","Type":"ContainerDied","Data":"a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125"} Jul 22 16:42:42.322633 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:42.322392 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd7b4788c-g2l9f" event={"ID":"1ce2cc23-bf63-4325-b06e-8b07d503ed49","Type":"ContainerDied","Data":"0d0bb572ab77a06cb9da37917c3fba87e65abb5acd7ba956351fc4834590bd56"} Jul 22 16:42:42.322633 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:42.322416 2581 scope.go:117] "RemoveContainer" containerID="a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125" Jul 22 16:42:42.331051 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:42.331030 2581 scope.go:117] "RemoveContainer" containerID="a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125" Jul 22 16:42:42.331289 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:42:42.331269 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125\": container with ID starting with a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125 not found: ID does not exist" containerID="a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125" Jul 22 16:42:42.331355 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:42.331300 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125"} err="failed to get container status \"a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125\": rpc error: code = NotFound desc = could not find container \"a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125\": container with ID starting with a88d6244878051d5ee408b0d5bd583fdeb9e6a8d88609fffcc9d8d26b96cd125 not found: ID does not exist" Jul 22 16:42:42.342799 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:42.342778 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd7b4788c-g2l9f"] Jul 22 16:42:42.346417 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:42.346399 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cd7b4788c-g2l9f"] Jul 22 16:42:43.617653 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:42:43.617625 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ce2cc23-bf63-4325-b06e-8b07d503ed49" path="/var/lib/kubelet/pods/1ce2cc23-bf63-4325-b06e-8b07d503ed49/volumes" Jul 22 16:43:07.250282 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.250200 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f74fd479d-dn26w"] Jul 22 16:43:07.250714 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.250524 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ce2cc23-bf63-4325-b06e-8b07d503ed49" containerName="console" Jul 22 16:43:07.250714 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.250541 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ce2cc23-bf63-4325-b06e-8b07d503ed49" containerName="console" Jul 22 16:43:07.250714 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.250567 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" containerName="registry" Jul 22 16:43:07.250714 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.250576 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" containerName="registry" Jul 22 16:43:07.250714 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.250658 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b9cee3b-a61a-4de1-9b7d-21d1f68b6b2a" containerName="registry" Jul 22 16:43:07.250714 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.250671 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ce2cc23-bf63-4325-b06e-8b07d503ed49" containerName="console" Jul 22 16:43:07.253916 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.253897 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.256342 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.256316 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 22 16:43:07.256613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.256580 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 22 16:43:07.256613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.256587 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-77ln9\"" Jul 22 16:43:07.256748 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.256703 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 22 16:43:07.257066 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.257049 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 22 16:43:07.257134 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.257121 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 22 16:43:07.262560 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.262540 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 22 16:43:07.273832 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.273811 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f74fd479d-dn26w"] Jul 22 16:43:07.381292 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.381265 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.381428 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.381296 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c333945b-4dac-4941-868d-a43d34fdff39-serving-certs-ca-bundle\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.381428 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.381350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c333945b-4dac-4941-868d-a43d34fdff39-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.381428 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.381392 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkdgv\" (UniqueName: \"kubernetes.io/projected/c333945b-4dac-4941-868d-a43d34fdff39-kube-api-access-wkdgv\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.381561 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.381439 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-secret-telemeter-client\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.381561 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.381472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-federate-client-tls\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.381561 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.381488 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-telemeter-client-tls\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.381561 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.381512 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c333945b-4dac-4941-868d-a43d34fdff39-metrics-client-ca\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.482595 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.482563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c333945b-4dac-4941-868d-a43d34fdff39-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.482711 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.482603 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkdgv\" (UniqueName: \"kubernetes.io/projected/c333945b-4dac-4941-868d-a43d34fdff39-kube-api-access-wkdgv\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.482711 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.482624 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-secret-telemeter-client\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.482711 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.482652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-federate-client-tls\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.482928 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.482903 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-telemeter-client-tls\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.482988 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.482949 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c333945b-4dac-4941-868d-a43d34fdff39-metrics-client-ca\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.483029 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.482995 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.483080 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.483025 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c333945b-4dac-4941-868d-a43d34fdff39-serving-certs-ca-bundle\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.483553 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.483518 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c333945b-4dac-4941-868d-a43d34fdff39-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.483728 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.483710 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c333945b-4dac-4941-868d-a43d34fdff39-serving-certs-ca-bundle\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.483821 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.483734 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c333945b-4dac-4941-868d-a43d34fdff39-metrics-client-ca\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.485151 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.485130 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-federate-client-tls\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.485248 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.485229 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-telemeter-client-tls\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.485506 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.485488 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.485591 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.485572 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c333945b-4dac-4941-868d-a43d34fdff39-secret-telemeter-client\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.490492 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.490473 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkdgv\" (UniqueName: \"kubernetes.io/projected/c333945b-4dac-4941-868d-a43d34fdff39-kube-api-access-wkdgv\") pod \"telemeter-client-7f74fd479d-dn26w\" (UID: \"c333945b-4dac-4941-868d-a43d34fdff39\") " pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.563142 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.563095 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" Jul 22 16:43:07.696499 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:07.696327 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f74fd479d-dn26w"] Jul 22 16:43:07.698614 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:43:07.698579 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc333945b_4dac_4941_868d_a43d34fdff39.slice/crio-31e1fca2b1f5281c1d074de47eeb75f2840cf1d65bed120ff9eee94de2950474 WatchSource:0}: Error finding container 31e1fca2b1f5281c1d074de47eeb75f2840cf1d65bed120ff9eee94de2950474: Status 404 returned error can't find the container with id 31e1fca2b1f5281c1d074de47eeb75f2840cf1d65bed120ff9eee94de2950474 Jul 22 16:43:08.419958 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:08.419921 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" event={"ID":"c333945b-4dac-4941-868d-a43d34fdff39","Type":"ContainerStarted","Data":"31e1fca2b1f5281c1d074de47eeb75f2840cf1d65bed120ff9eee94de2950474"} Jul 22 16:43:10.430943 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:10.430906 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" event={"ID":"c333945b-4dac-4941-868d-a43d34fdff39","Type":"ContainerStarted","Data":"1c79b417ca7fd053d63504cbde14df3144b72617da81bf21f391bc92204ba3c5"} Jul 22 16:43:11.437133 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:11.437098 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" event={"ID":"c333945b-4dac-4941-868d-a43d34fdff39","Type":"ContainerStarted","Data":"8b17895b05e6521a567eaedd47c336f44b4dfe4f3c0f9f987490dd9a5b8ccefd"} Jul 22 16:43:11.437520 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:11.437149 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" event={"ID":"c333945b-4dac-4941-868d-a43d34fdff39","Type":"ContainerStarted","Data":"e429e3eb341fbc36bf9d9dc75d3b2279f21872bfdfd0063a85fa197849917bc9"} Jul 22 16:43:11.459243 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:11.459204 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f74fd479d-dn26w" podStartSLOduration=1.499446611 podStartE2EDuration="4.459191029s" totalImagesPullingTime="2.959744418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:43:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:43:11.458908163 +0000 UTC m=+178.369565338" watchObservedRunningTime="2026-07-22 16:43:11.459191029 +0000 UTC m=+178.369848201" Jul 22 16:43:12.583184 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.583148 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8566bd7999-9mvrp"] Jul 22 16:43:12.586411 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.586361 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.601439 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.601415 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8566bd7999-9mvrp"] Jul 22 16:43:12.618270 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.618242 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-config\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.618357 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.618280 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-service-ca\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.618357 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.618296 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-oauth-serving-cert\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.618454 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.618389 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-557zk\" (UniqueName: \"kubernetes.io/projected/5ff94068-b11c-4f53-8814-e11a3e6d672a-kube-api-access-557zk\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.618454 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.618440 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-oauth-config\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.618560 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.618521 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-serving-cert\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.618620 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.618571 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-trusted-ca-bundle\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.719235 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.719206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-557zk\" (UniqueName: \"kubernetes.io/projected/5ff94068-b11c-4f53-8814-e11a3e6d672a-kube-api-access-557zk\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.719327 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.719245 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-oauth-config\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.719327 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.719269 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-serving-cert\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.719327 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.719287 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-trusted-ca-bundle\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.719327 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.719309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-config\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.719565 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.719336 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-service-ca\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.719565 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.719350 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-oauth-serving-cert\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.720082 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.720058 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-service-ca\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.720167 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.720094 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-config\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.720167 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.720147 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-oauth-serving-cert\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.720349 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.720326 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-trusted-ca-bundle\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.721569 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.721553 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-oauth-config\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.721924 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.721905 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-serving-cert\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.728000 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.727977 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-557zk\" (UniqueName: \"kubernetes.io/projected/5ff94068-b11c-4f53-8814-e11a3e6d672a-kube-api-access-557zk\") pod \"console-8566bd7999-9mvrp\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:12.896685 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:12.896623 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:13.026498 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:13.026469 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8566bd7999-9mvrp"] Jul 22 16:43:13.029456 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:43:13.029425 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff94068_b11c_4f53_8814_e11a3e6d672a.slice/crio-66a6f1893f15d03bec90f7010f56b1e4856fc1126f27fc76b45f4b26ab003e45 WatchSource:0}: Error finding container 66a6f1893f15d03bec90f7010f56b1e4856fc1126f27fc76b45f4b26ab003e45: Status 404 returned error can't find the container with id 66a6f1893f15d03bec90f7010f56b1e4856fc1126f27fc76b45f4b26ab003e45 Jul 22 16:43:13.446433 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:13.446396 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8566bd7999-9mvrp" event={"ID":"5ff94068-b11c-4f53-8814-e11a3e6d672a","Type":"ContainerStarted","Data":"5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488"} Jul 22 16:43:13.446612 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:13.446446 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8566bd7999-9mvrp" event={"ID":"5ff94068-b11c-4f53-8814-e11a3e6d672a","Type":"ContainerStarted","Data":"66a6f1893f15d03bec90f7010f56b1e4856fc1126f27fc76b45f4b26ab003e45"} Jul 22 16:43:13.468680 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:13.468639 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8566bd7999-9mvrp" podStartSLOduration=1.468624067 podStartE2EDuration="1.468624067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:43:13.467081177 +0000 UTC m=+180.377738383" watchObservedRunningTime="2026-07-22 16:43:13.468624067 +0000 UTC m=+180.379281238" Jul 22 16:43:22.897421 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:22.897376 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:22.897824 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:22.897442 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:22.902215 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:22.902195 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:23.487492 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:23.487460 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:43:23.551009 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:23.550973 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bbd447985-9xb5h"] Jul 22 16:43:48.573632 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.573568 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bbd447985-9xb5h" podUID="790a6b95-011b-428e-9910-9fa99a0c19df" containerName="console" containerID="cri-o://a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73" gracePeriod=15 Jul 22 16:43:48.811447 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.811424 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:43:48.875238 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875156 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-service-ca\") pod \"790a6b95-011b-428e-9910-9fa99a0c19df\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " Jul 22 16:43:48.875238 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875219 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-console-config\") pod \"790a6b95-011b-428e-9910-9fa99a0c19df\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " Jul 22 16:43:48.875417 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875324 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-oauth-serving-cert\") pod \"790a6b95-011b-428e-9910-9fa99a0c19df\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " Jul 22 16:43:48.875417 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875377 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-serving-cert\") pod \"790a6b95-011b-428e-9910-9fa99a0c19df\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " Jul 22 16:43:48.875504 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875431 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfnxs\" (UniqueName: \"kubernetes.io/projected/790a6b95-011b-428e-9910-9fa99a0c19df-kube-api-access-hfnxs\") pod \"790a6b95-011b-428e-9910-9fa99a0c19df\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " Jul 22 16:43:48.875504 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875455 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-trusted-ca-bundle\") pod \"790a6b95-011b-428e-9910-9fa99a0c19df\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " Jul 22 16:43:48.875504 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875489 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-oauth-config\") pod \"790a6b95-011b-428e-9910-9fa99a0c19df\" (UID: \"790a6b95-011b-428e-9910-9fa99a0c19df\") " Jul 22 16:43:48.875650 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875562 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-console-config" (OuterVolumeSpecName: "console-config") pod "790a6b95-011b-428e-9910-9fa99a0c19df" (UID: "790a6b95-011b-428e-9910-9fa99a0c19df"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:43:48.875650 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875604 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-service-ca" (OuterVolumeSpecName: "service-ca") pod "790a6b95-011b-428e-9910-9fa99a0c19df" (UID: "790a6b95-011b-428e-9910-9fa99a0c19df"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:43:48.875764 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875667 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "790a6b95-011b-428e-9910-9fa99a0c19df" (UID: "790a6b95-011b-428e-9910-9fa99a0c19df"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:43:48.875764 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875752 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:43:48.875865 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875770 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-console-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:43:48.875865 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.875785 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-oauth-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:43:48.876042 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.876023 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "790a6b95-011b-428e-9910-9fa99a0c19df" (UID: "790a6b95-011b-428e-9910-9fa99a0c19df"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:43:48.877473 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.877432 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "790a6b95-011b-428e-9910-9fa99a0c19df" (UID: "790a6b95-011b-428e-9910-9fa99a0c19df"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:43:48.877558 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.877519 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/790a6b95-011b-428e-9910-9fa99a0c19df-kube-api-access-hfnxs" (OuterVolumeSpecName: "kube-api-access-hfnxs") pod "790a6b95-011b-428e-9910-9fa99a0c19df" (UID: "790a6b95-011b-428e-9910-9fa99a0c19df"). InnerVolumeSpecName "kube-api-access-hfnxs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:43:48.877924 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.877908 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "790a6b95-011b-428e-9910-9fa99a0c19df" (UID: "790a6b95-011b-428e-9910-9fa99a0c19df"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:43:48.976356 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.976328 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:43:48.976356 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.976353 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfnxs\" (UniqueName: \"kubernetes.io/projected/790a6b95-011b-428e-9910-9fa99a0c19df-kube-api-access-hfnxs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:43:48.976356 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.976362 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790a6b95-011b-428e-9910-9fa99a0c19df-trusted-ca-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:43:48.976557 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:48.976387 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/790a6b95-011b-428e-9910-9fa99a0c19df-console-oauth-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:43:49.588435 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:49.588403 2581 generic.go:356] "Generic (PLEG): container finished" podID="790a6b95-011b-428e-9910-9fa99a0c19df" containerID="a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73" exitCode=0 Jul 22 16:43:49.588810 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:49.588442 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bbd447985-9xb5h" event={"ID":"790a6b95-011b-428e-9910-9fa99a0c19df","Type":"ContainerDied","Data":"a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73"} Jul 22 16:43:49.588810 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:49.588446 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bbd447985-9xb5h" Jul 22 16:43:49.588810 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:49.588477 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bbd447985-9xb5h" event={"ID":"790a6b95-011b-428e-9910-9fa99a0c19df","Type":"ContainerDied","Data":"10ebeafb8824b028b471fa75ebb5bfd8b3c35c3c2ab1e5f2ea0691f1b30fbe59"} Jul 22 16:43:49.588810 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:49.588495 2581 scope.go:117] "RemoveContainer" containerID="a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73" Jul 22 16:43:49.597965 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:49.597954 2581 scope.go:117] "RemoveContainer" containerID="a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73" Jul 22 16:43:49.598198 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:43:49.598178 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73\": container with ID starting with a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73 not found: ID does not exist" containerID="a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73" Jul 22 16:43:49.598254 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:49.598206 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73"} err="failed to get container status \"a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73\": rpc error: code = NotFound desc = could not find container \"a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73\": container with ID starting with a8c5dd54e7a6919d9ea555edad873e5dcd28449eb7e925691d5252d3cf689f73 not found: ID does not exist" Jul 22 16:43:49.610092 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:49.610065 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bbd447985-9xb5h"] Jul 22 16:43:49.617748 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:49.617727 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bbd447985-9xb5h"] Jul 22 16:43:51.617254 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:43:51.617222 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="790a6b95-011b-428e-9910-9fa99a0c19df" path="/var/lib/kubelet/pods/790a6b95-011b-428e-9910-9fa99a0c19df/volumes" Jul 22 16:44:18.021958 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.021923 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8679dbbccf-84r4r"] Jul 22 16:44:18.022498 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.022320 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="790a6b95-011b-428e-9910-9fa99a0c19df" containerName="console" Jul 22 16:44:18.022498 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.022335 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="790a6b95-011b-428e-9910-9fa99a0c19df" containerName="console" Jul 22 16:44:18.022498 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.022440 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="790a6b95-011b-428e-9910-9fa99a0c19df" containerName="console" Jul 22 16:44:18.026712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.026692 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.040995 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.040973 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8679dbbccf-84r4r"] Jul 22 16:44:18.169563 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.169529 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-config\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.169702 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.169572 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-trusted-ca-bundle\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.169702 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.169621 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzcd4\" (UniqueName: \"kubernetes.io/projected/a4fbee2d-aa35-4d85-9dea-bad849069b17-kube-api-access-kzcd4\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.169702 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.169651 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-service-ca\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.169804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.169697 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-serving-cert\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.169804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.169758 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-oauth-config\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.169804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.169776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-oauth-serving-cert\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.270325 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.270303 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-trusted-ca-bundle\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.270453 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.270339 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzcd4\" (UniqueName: \"kubernetes.io/projected/a4fbee2d-aa35-4d85-9dea-bad849069b17-kube-api-access-kzcd4\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.270453 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.270360 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-service-ca\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.270453 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.270394 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-serving-cert\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.270453 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.270436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-oauth-config\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.270453 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.270451 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-oauth-serving-cert\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.270708 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.270471 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-config\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.271145 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.271119 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-service-ca\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.271248 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.271181 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-oauth-serving-cert\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.271248 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.271221 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-config\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.271406 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.271386 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-trusted-ca-bundle\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.272930 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.272872 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-oauth-config\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.272998 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.272984 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-serving-cert\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.279106 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.279084 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzcd4\" (UniqueName: \"kubernetes.io/projected/a4fbee2d-aa35-4d85-9dea-bad849069b17-kube-api-access-kzcd4\") pod \"console-8679dbbccf-84r4r\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.335865 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.335842 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:18.461454 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.461428 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8679dbbccf-84r4r"] Jul 22 16:44:18.463949 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:44:18.463922 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4fbee2d_aa35_4d85_9dea_bad849069b17.slice/crio-709958fff21eb1aefd6660f29629e7fc0048f257a517957754ad02be68b0ca4c WatchSource:0}: Error finding container 709958fff21eb1aefd6660f29629e7fc0048f257a517957754ad02be68b0ca4c: Status 404 returned error can't find the container with id 709958fff21eb1aefd6660f29629e7fc0048f257a517957754ad02be68b0ca4c Jul 22 16:44:18.691003 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.690914 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8679dbbccf-84r4r" event={"ID":"a4fbee2d-aa35-4d85-9dea-bad849069b17","Type":"ContainerStarted","Data":"8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1"} Jul 22 16:44:18.691003 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.690966 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8679dbbccf-84r4r" event={"ID":"a4fbee2d-aa35-4d85-9dea-bad849069b17","Type":"ContainerStarted","Data":"709958fff21eb1aefd6660f29629e7fc0048f257a517957754ad02be68b0ca4c"} Jul 22 16:44:18.709956 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:18.709909 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8679dbbccf-84r4r" podStartSLOduration=0.70989425 podStartE2EDuration="709.89425ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:44:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:44:18.707911084 +0000 UTC m=+245.618568274" watchObservedRunningTime="2026-07-22 16:44:18.70989425 +0000 UTC m=+245.620551422" Jul 22 16:44:28.336598 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:28.336567 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:28.337100 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:28.336624 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:28.340970 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:28.340949 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:28.732199 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:28.732169 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:44:28.778758 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:28.778725 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8566bd7999-9mvrp"] Jul 22 16:44:53.800845 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:53.800803 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8566bd7999-9mvrp" podUID="5ff94068-b11c-4f53-8814-e11a3e6d672a" containerName="console" containerID="cri-o://5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488" gracePeriod=15 Jul 22 16:44:54.038708 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.038686 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:44:54.120540 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.120474 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-557zk\" (UniqueName: \"kubernetes.io/projected/5ff94068-b11c-4f53-8814-e11a3e6d672a-kube-api-access-557zk\") pod \"5ff94068-b11c-4f53-8814-e11a3e6d672a\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " Jul 22 16:44:54.120540 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.120512 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-config\") pod \"5ff94068-b11c-4f53-8814-e11a3e6d672a\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " Jul 22 16:44:54.120540 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.120530 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-service-ca\") pod \"5ff94068-b11c-4f53-8814-e11a3e6d672a\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " Jul 22 16:44:54.120716 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.120547 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-trusted-ca-bundle\") pod \"5ff94068-b11c-4f53-8814-e11a3e6d672a\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " Jul 22 16:44:54.120716 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.120579 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-oauth-config\") pod \"5ff94068-b11c-4f53-8814-e11a3e6d672a\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " Jul 22 16:44:54.120716 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.120615 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-oauth-serving-cert\") pod \"5ff94068-b11c-4f53-8814-e11a3e6d672a\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " Jul 22 16:44:54.120716 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.120645 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-serving-cert\") pod \"5ff94068-b11c-4f53-8814-e11a3e6d672a\" (UID: \"5ff94068-b11c-4f53-8814-e11a3e6d672a\") " Jul 22 16:44:54.120945 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.120917 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-config" (OuterVolumeSpecName: "console-config") pod "5ff94068-b11c-4f53-8814-e11a3e6d672a" (UID: "5ff94068-b11c-4f53-8814-e11a3e6d672a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:44:54.121011 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.120945 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-service-ca" (OuterVolumeSpecName: "service-ca") pod "5ff94068-b11c-4f53-8814-e11a3e6d672a" (UID: "5ff94068-b11c-4f53-8814-e11a3e6d672a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:44:54.121073 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.120968 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5ff94068-b11c-4f53-8814-e11a3e6d672a" (UID: "5ff94068-b11c-4f53-8814-e11a3e6d672a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:44:54.121128 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.121110 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5ff94068-b11c-4f53-8814-e11a3e6d672a" (UID: "5ff94068-b11c-4f53-8814-e11a3e6d672a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:44:54.122770 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.122749 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5ff94068-b11c-4f53-8814-e11a3e6d672a" (UID: "5ff94068-b11c-4f53-8814-e11a3e6d672a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:44:54.122850 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.122792 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5ff94068-b11c-4f53-8814-e11a3e6d672a" (UID: "5ff94068-b11c-4f53-8814-e11a3e6d672a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:44:54.122897 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.122883 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff94068-b11c-4f53-8814-e11a3e6d672a-kube-api-access-557zk" (OuterVolumeSpecName: "kube-api-access-557zk") pod "5ff94068-b11c-4f53-8814-e11a3e6d672a" (UID: "5ff94068-b11c-4f53-8814-e11a3e6d672a"). InnerVolumeSpecName "kube-api-access-557zk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:44:54.221202 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.221177 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-557zk\" (UniqueName: \"kubernetes.io/projected/5ff94068-b11c-4f53-8814-e11a3e6d672a-kube-api-access-557zk\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:44:54.221202 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.221199 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:44:54.221202 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.221209 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:44:54.221402 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.221217 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-trusted-ca-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:44:54.221402 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.221225 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-oauth-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:44:54.221402 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.221233 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ff94068-b11c-4f53-8814-e11a3e6d672a-oauth-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:44:54.221402 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.221242 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff94068-b11c-4f53-8814-e11a3e6d672a-console-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:44:54.827103 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.827072 2581 generic.go:356] "Generic (PLEG): container finished" podID="5ff94068-b11c-4f53-8814-e11a3e6d672a" containerID="5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488" exitCode=0 Jul 22 16:44:54.827505 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.827129 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8566bd7999-9mvrp" Jul 22 16:44:54.827505 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.827157 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8566bd7999-9mvrp" event={"ID":"5ff94068-b11c-4f53-8814-e11a3e6d672a","Type":"ContainerDied","Data":"5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488"} Jul 22 16:44:54.827505 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.827209 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8566bd7999-9mvrp" event={"ID":"5ff94068-b11c-4f53-8814-e11a3e6d672a","Type":"ContainerDied","Data":"66a6f1893f15d03bec90f7010f56b1e4856fc1126f27fc76b45f4b26ab003e45"} Jul 22 16:44:54.827505 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.827229 2581 scope.go:117] "RemoveContainer" containerID="5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488" Jul 22 16:44:54.836435 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.836418 2581 scope.go:117] "RemoveContainer" containerID="5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488" Jul 22 16:44:54.836647 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:44:54.836630 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488\": container with ID starting with 5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488 not found: ID does not exist" containerID="5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488" Jul 22 16:44:54.836695 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.836655 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488"} err="failed to get container status \"5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488\": rpc error: code = NotFound desc = could not find container \"5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488\": container with ID starting with 5a631a283fcc655abe1128734155f7c9789b074a137eced5b8660316ec3c8488 not found: ID does not exist" Jul 22 16:44:54.847959 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.847937 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8566bd7999-9mvrp"] Jul 22 16:44:54.854419 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:54.854398 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8566bd7999-9mvrp"] Jul 22 16:44:55.617128 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:44:55.617095 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ff94068-b11c-4f53-8814-e11a3e6d672a" path="/var/lib/kubelet/pods/5ff94068-b11c-4f53-8814-e11a3e6d672a/volumes" Jul 22 16:45:13.510717 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:13.510683 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 16:45:13.524717 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:13.524683 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 16:45:13.526024 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:13.525997 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 16:45:40.703848 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.703807 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp"] Jul 22 16:45:40.706205 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.704113 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ff94068-b11c-4f53-8814-e11a3e6d672a" containerName="console" Jul 22 16:45:40.706205 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.704125 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ff94068-b11c-4f53-8814-e11a3e6d672a" containerName="console" Jul 22 16:45:40.706205 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.704188 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ff94068-b11c-4f53-8814-e11a3e6d672a" containerName="console" Jul 22 16:45:40.707173 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.707157 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:40.709943 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.709913 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 16:45:40.710058 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.709969 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhhz7\"" Jul 22 16:45:40.710835 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.710818 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 16:45:40.717065 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.717044 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp"] Jul 22 16:45:40.840593 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.840567 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:40.840713 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.840622 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7qtb\" (UniqueName: \"kubernetes.io/projected/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-kube-api-access-q7qtb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:40.840713 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.840652 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:40.941593 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.941565 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:40.941672 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.941616 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7qtb\" (UniqueName: \"kubernetes.io/projected/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-kube-api-access-q7qtb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:40.941712 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.941670 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:40.941943 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.941922 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:40.941979 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.941962 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:40.950051 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:40.950029 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7qtb\" (UniqueName: \"kubernetes.io/projected/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-kube-api-access-q7qtb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:41.016421 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:41.016399 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:45:41.141080 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:41.141049 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp"] Jul 22 16:45:41.143912 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:45:41.143884 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2a07d78_4651_4662_96c7_e9fe3b8e18d2.slice/crio-e5e90e99e434ac8cf6d922fc272715090372068edac0c5d05588bebe88cda48c WatchSource:0}: Error finding container e5e90e99e434ac8cf6d922fc272715090372068edac0c5d05588bebe88cda48c: Status 404 returned error can't find the container with id e5e90e99e434ac8cf6d922fc272715090372068edac0c5d05588bebe88cda48c Jul 22 16:45:41.145864 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:41.145841 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:45:42.003906 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:42.003870 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" event={"ID":"e2a07d78-4651-4662-96c7-e9fe3b8e18d2","Type":"ContainerStarted","Data":"e5e90e99e434ac8cf6d922fc272715090372068edac0c5d05588bebe88cda48c"} Jul 22 16:45:48.030945 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:48.030909 2581 generic.go:356] "Generic (PLEG): container finished" podID="e2a07d78-4651-4662-96c7-e9fe3b8e18d2" containerID="cb3da7ccd265f74235c4d696b4c7bc97bb7c8ec42cd8ffbcf7063785ef19544e" exitCode=0 Jul 22 16:45:48.031380 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:48.030994 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" event={"ID":"e2a07d78-4651-4662-96c7-e9fe3b8e18d2","Type":"ContainerDied","Data":"cb3da7ccd265f74235c4d696b4c7bc97bb7c8ec42cd8ffbcf7063785ef19544e"} Jul 22 16:45:52.047042 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:52.046975 2581 generic.go:356] "Generic (PLEG): container finished" podID="e2a07d78-4651-4662-96c7-e9fe3b8e18d2" containerID="5f91c72dd99d7e7fefdf2afb51a7163e15f51f06b021a46e4aef4f9acb3f573d" exitCode=0 Jul 22 16:45:52.047324 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:45:52.047043 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" event={"ID":"e2a07d78-4651-4662-96c7-e9fe3b8e18d2","Type":"ContainerDied","Data":"5f91c72dd99d7e7fefdf2afb51a7163e15f51f06b021a46e4aef4f9acb3f573d"} Jul 22 16:46:00.081829 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:00.081791 2581 generic.go:356] "Generic (PLEG): container finished" podID="e2a07d78-4651-4662-96c7-e9fe3b8e18d2" containerID="95833be56708cba5d1c2aebabdd84fd8bd45c562314c982132fe0e8a0c60cf36" exitCode=0 Jul 22 16:46:00.082216 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:00.081842 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" event={"ID":"e2a07d78-4651-4662-96c7-e9fe3b8e18d2","Type":"ContainerDied","Data":"95833be56708cba5d1c2aebabdd84fd8bd45c562314c982132fe0e8a0c60cf36"} Jul 22 16:46:01.252340 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:01.252312 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:46:01.406520 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:01.406456 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-util\") pod \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " Jul 22 16:46:01.406520 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:01.406491 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7qtb\" (UniqueName: \"kubernetes.io/projected/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-kube-api-access-q7qtb\") pod \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " Jul 22 16:46:01.406719 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:01.406545 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-bundle\") pod \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\" (UID: \"e2a07d78-4651-4662-96c7-e9fe3b8e18d2\") " Jul 22 16:46:01.407144 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:01.407123 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-bundle" (OuterVolumeSpecName: "bundle") pod "e2a07d78-4651-4662-96c7-e9fe3b8e18d2" (UID: "e2a07d78-4651-4662-96c7-e9fe3b8e18d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:01.408619 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:01.408582 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-kube-api-access-q7qtb" (OuterVolumeSpecName: "kube-api-access-q7qtb") pod "e2a07d78-4651-4662-96c7-e9fe3b8e18d2" (UID: "e2a07d78-4651-4662-96c7-e9fe3b8e18d2"). InnerVolumeSpecName "kube-api-access-q7qtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:46:01.411599 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:01.411579 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-util" (OuterVolumeSpecName: "util") pod "e2a07d78-4651-4662-96c7-e9fe3b8e18d2" (UID: "e2a07d78-4651-4662-96c7-e9fe3b8e18d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:01.507597 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:01.507571 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:46:01.507597 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:01.507596 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7qtb\" (UniqueName: \"kubernetes.io/projected/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-kube-api-access-q7qtb\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:46:01.507722 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:01.507607 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2a07d78-4651-4662-96c7-e9fe3b8e18d2-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:46:02.091924 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:02.091892 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" event={"ID":"e2a07d78-4651-4662-96c7-e9fe3b8e18d2","Type":"ContainerDied","Data":"e5e90e99e434ac8cf6d922fc272715090372068edac0c5d05588bebe88cda48c"} Jul 22 16:46:02.092062 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:02.091928 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402cjrp" Jul 22 16:46:02.092062 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:02.091936 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5e90e99e434ac8cf6d922fc272715090372068edac0c5d05588bebe88cda48c" Jul 22 16:46:09.807387 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.807294 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm"] Jul 22 16:46:09.807808 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.807633 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2a07d78-4651-4662-96c7-e9fe3b8e18d2" containerName="pull" Jul 22 16:46:09.807808 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.807647 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2a07d78-4651-4662-96c7-e9fe3b8e18d2" containerName="pull" Jul 22 16:46:09.807808 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.807666 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2a07d78-4651-4662-96c7-e9fe3b8e18d2" containerName="util" Jul 22 16:46:09.807808 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.807671 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2a07d78-4651-4662-96c7-e9fe3b8e18d2" containerName="util" Jul 22 16:46:09.807808 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.807683 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2a07d78-4651-4662-96c7-e9fe3b8e18d2" containerName="extract" Jul 22 16:46:09.807808 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.807689 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2a07d78-4651-4662-96c7-e9fe3b8e18d2" containerName="extract" Jul 22 16:46:09.807808 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.807736 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2a07d78-4651-4662-96c7-e9fe3b8e18d2" containerName="extract" Jul 22 16:46:09.810627 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.810612 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" Jul 22 16:46:09.816213 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.816188 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 22 16:46:09.816346 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.816230 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:46:09.817613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.817598 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-f55q4\"" Jul 22 16:46:09.835936 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.835907 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm"] Jul 22 16:46:09.966770 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.966743 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qclbp\" (UniqueName: \"kubernetes.io/projected/2af08d6c-cfa5-453d-afbd-87b9f6a5515a-kube-api-access-qclbp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nswm\" (UID: \"2af08d6c-cfa5-453d-afbd-87b9f6a5515a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" Jul 22 16:46:09.966867 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:09.966774 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2af08d6c-cfa5-453d-afbd-87b9f6a5515a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nswm\" (UID: \"2af08d6c-cfa5-453d-afbd-87b9f6a5515a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" Jul 22 16:46:10.067843 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:10.067787 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qclbp\" (UniqueName: \"kubernetes.io/projected/2af08d6c-cfa5-453d-afbd-87b9f6a5515a-kube-api-access-qclbp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nswm\" (UID: \"2af08d6c-cfa5-453d-afbd-87b9f6a5515a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" Jul 22 16:46:10.067843 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:10.067817 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2af08d6c-cfa5-453d-afbd-87b9f6a5515a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nswm\" (UID: \"2af08d6c-cfa5-453d-afbd-87b9f6a5515a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" Jul 22 16:46:10.068132 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:10.068117 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2af08d6c-cfa5-453d-afbd-87b9f6a5515a-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nswm\" (UID: \"2af08d6c-cfa5-453d-afbd-87b9f6a5515a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" Jul 22 16:46:10.076635 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:10.076606 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qclbp\" (UniqueName: \"kubernetes.io/projected/2af08d6c-cfa5-453d-afbd-87b9f6a5515a-kube-api-access-qclbp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6nswm\" (UID: \"2af08d6c-cfa5-453d-afbd-87b9f6a5515a\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" Jul 22 16:46:10.118788 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:10.118766 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" Jul 22 16:46:10.265772 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:10.265746 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm"] Jul 22 16:46:10.267350 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:46:10.267322 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af08d6c_cfa5_453d_afbd_87b9f6a5515a.slice/crio-b2a4a462f47643e7af65ca776971f302b2788ba347a2df78db7ea7bf0d15d04f WatchSource:0}: Error finding container b2a4a462f47643e7af65ca776971f302b2788ba347a2df78db7ea7bf0d15d04f: Status 404 returned error can't find the container with id b2a4a462f47643e7af65ca776971f302b2788ba347a2df78db7ea7bf0d15d04f Jul 22 16:46:11.126206 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:11.126172 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" event={"ID":"2af08d6c-cfa5-453d-afbd-87b9f6a5515a","Type":"ContainerStarted","Data":"b2a4a462f47643e7af65ca776971f302b2788ba347a2df78db7ea7bf0d15d04f"} Jul 22 16:46:13.136234 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:13.136195 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" event={"ID":"2af08d6c-cfa5-453d-afbd-87b9f6a5515a","Type":"ContainerStarted","Data":"c6d6694021abd3c364ca21a98d8e000637da4ac4a3587f1ab7287f638a83a2c8"} Jul 22 16:46:13.196928 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:13.196829 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6nswm" podStartSLOduration=1.544035751 podStartE2EDuration="4.196814483s" totalImagesPullingTime="2.652778732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:46:13.195580535 +0000 UTC m=+360.106237721" watchObservedRunningTime="2026-07-22 16:46:13.196814483 +0000 UTC m=+360.107471656" Jul 22 16:46:14.353422 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.353388 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56"] Jul 22 16:46:14.356809 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.356793 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.359788 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.359754 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhhz7\"" Jul 22 16:46:14.359958 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.359946 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 16:46:14.360562 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.360540 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 16:46:14.383021 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.383003 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56"] Jul 22 16:46:14.505453 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.505426 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.505588 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.505481 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6d72\" (UniqueName: \"kubernetes.io/projected/e4bef988-f09d-4f8a-b55b-20f1c194929a-kube-api-access-l6d72\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.505588 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.505542 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.606347 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.606280 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6d72\" (UniqueName: \"kubernetes.io/projected/e4bef988-f09d-4f8a-b55b-20f1c194929a-kube-api-access-l6d72\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.606347 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.606311 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.606347 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.606348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.606705 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.606682 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.606761 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.606719 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.647508 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.647484 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6d72\" (UniqueName: \"kubernetes.io/projected/e4bef988-f09d-4f8a-b55b-20f1c194929a-kube-api-access-l6d72\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.665274 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.665243 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:14.814838 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:14.814808 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56"] Jul 22 16:46:14.816333 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:46:14.816311 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4bef988_f09d_4f8a_b55b_20f1c194929a.slice/crio-b62f821ecd59482b0e967c73ab6b3cabd44bfff701d321b9b580585b03163014 WatchSource:0}: Error finding container b62f821ecd59482b0e967c73ab6b3cabd44bfff701d321b9b580585b03163014: Status 404 returned error can't find the container with id b62f821ecd59482b0e967c73ab6b3cabd44bfff701d321b9b580585b03163014 Jul 22 16:46:15.145290 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:15.145221 2581 generic.go:356] "Generic (PLEG): container finished" podID="e4bef988-f09d-4f8a-b55b-20f1c194929a" containerID="6ab2b515cc9b2fceaef460a6040903896618cecf7a65b9edbc7c8913545171f5" exitCode=0 Jul 22 16:46:15.145425 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:15.145301 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" event={"ID":"e4bef988-f09d-4f8a-b55b-20f1c194929a","Type":"ContainerDied","Data":"6ab2b515cc9b2fceaef460a6040903896618cecf7a65b9edbc7c8913545171f5"} Jul 22 16:46:15.145425 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:15.145350 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" event={"ID":"e4bef988-f09d-4f8a-b55b-20f1c194929a","Type":"ContainerStarted","Data":"b62f821ecd59482b0e967c73ab6b3cabd44bfff701d321b9b580585b03163014"} Jul 22 16:46:16.741523 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:16.741493 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tgj97"] Jul 22 16:46:16.744902 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:16.744877 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" Jul 22 16:46:16.748271 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:16.748245 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 16:46:16.748397 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:16.748346 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 16:46:16.749330 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:16.749305 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-nx9bs\"" Jul 22 16:46:16.761739 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:16.761716 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tgj97"] Jul 22 16:46:16.923686 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:16.923655 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/712bb46d-d73d-4c56-a3c1-509cd0572a99-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tgj97\" (UID: \"712bb46d-d73d-4c56-a3c1-509cd0572a99\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" Jul 22 16:46:16.923859 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:16.923704 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khhg6\" (UniqueName: \"kubernetes.io/projected/712bb46d-d73d-4c56-a3c1-509cd0572a99-kube-api-access-khhg6\") pod \"cert-manager-cainjector-559cdd8c87-tgj97\" (UID: \"712bb46d-d73d-4c56-a3c1-509cd0572a99\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" Jul 22 16:46:17.025006 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:17.024920 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/712bb46d-d73d-4c56-a3c1-509cd0572a99-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tgj97\" (UID: \"712bb46d-d73d-4c56-a3c1-509cd0572a99\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" Jul 22 16:46:17.025006 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:17.024993 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khhg6\" (UniqueName: \"kubernetes.io/projected/712bb46d-d73d-4c56-a3c1-509cd0572a99-kube-api-access-khhg6\") pod \"cert-manager-cainjector-559cdd8c87-tgj97\" (UID: \"712bb46d-d73d-4c56-a3c1-509cd0572a99\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" Jul 22 16:46:17.059114 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:17.059085 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/712bb46d-d73d-4c56-a3c1-509cd0572a99-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tgj97\" (UID: \"712bb46d-d73d-4c56-a3c1-509cd0572a99\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" Jul 22 16:46:17.059705 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:17.059677 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khhg6\" (UniqueName: \"kubernetes.io/projected/712bb46d-d73d-4c56-a3c1-509cd0572a99-kube-api-access-khhg6\") pod \"cert-manager-cainjector-559cdd8c87-tgj97\" (UID: \"712bb46d-d73d-4c56-a3c1-509cd0572a99\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" Jul 22 16:46:17.069433 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:17.069411 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" Jul 22 16:46:17.565867 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:17.565812 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tgj97"] Jul 22 16:46:17.568238 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:46:17.568212 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod712bb46d_d73d_4c56_a3c1_509cd0572a99.slice/crio-840e90e0595be238af45d8c004b3c4edc19f043af2419ca9f8aa412769bfc61e WatchSource:0}: Error finding container 840e90e0595be238af45d8c004b3c4edc19f043af2419ca9f8aa412769bfc61e: Status 404 returned error can't find the container with id 840e90e0595be238af45d8c004b3c4edc19f043af2419ca9f8aa412769bfc61e Jul 22 16:46:18.159318 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:18.159288 2581 generic.go:356] "Generic (PLEG): container finished" podID="e4bef988-f09d-4f8a-b55b-20f1c194929a" containerID="648e074a59be566d62ac7145280afa46046360331cc72bfae350af08394776ab" exitCode=0 Jul 22 16:46:18.159762 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:18.159400 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" event={"ID":"e4bef988-f09d-4f8a-b55b-20f1c194929a","Type":"ContainerDied","Data":"648e074a59be566d62ac7145280afa46046360331cc72bfae350af08394776ab"} Jul 22 16:46:18.160900 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:18.160876 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" event={"ID":"712bb46d-d73d-4c56-a3c1-509cd0572a99","Type":"ContainerStarted","Data":"840e90e0595be238af45d8c004b3c4edc19f043af2419ca9f8aa412769bfc61e"} Jul 22 16:46:19.167524 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:19.167494 2581 generic.go:356] "Generic (PLEG): container finished" podID="e4bef988-f09d-4f8a-b55b-20f1c194929a" containerID="bc152b46b895673ac9c6e7d2bcda79275df50e5290ceb07c9a6207c7489afb09" exitCode=0 Jul 22 16:46:19.167977 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:19.167526 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" event={"ID":"e4bef988-f09d-4f8a-b55b-20f1c194929a","Type":"ContainerDied","Data":"bc152b46b895673ac9c6e7d2bcda79275df50e5290ceb07c9a6207c7489afb09"} Jul 22 16:46:20.306186 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:20.306166 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:20.452510 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:20.452439 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-util\") pod \"e4bef988-f09d-4f8a-b55b-20f1c194929a\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " Jul 22 16:46:20.452510 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:20.452488 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6d72\" (UniqueName: \"kubernetes.io/projected/e4bef988-f09d-4f8a-b55b-20f1c194929a-kube-api-access-l6d72\") pod \"e4bef988-f09d-4f8a-b55b-20f1c194929a\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " Jul 22 16:46:20.452733 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:20.452550 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-bundle\") pod \"e4bef988-f09d-4f8a-b55b-20f1c194929a\" (UID: \"e4bef988-f09d-4f8a-b55b-20f1c194929a\") " Jul 22 16:46:20.452925 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:20.452901 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-bundle" (OuterVolumeSpecName: "bundle") pod "e4bef988-f09d-4f8a-b55b-20f1c194929a" (UID: "e4bef988-f09d-4f8a-b55b-20f1c194929a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:20.454486 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:20.454466 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4bef988-f09d-4f8a-b55b-20f1c194929a-kube-api-access-l6d72" (OuterVolumeSpecName: "kube-api-access-l6d72") pod "e4bef988-f09d-4f8a-b55b-20f1c194929a" (UID: "e4bef988-f09d-4f8a-b55b-20f1c194929a"). InnerVolumeSpecName "kube-api-access-l6d72". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:46:20.490502 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:20.490426 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-util" (OuterVolumeSpecName: "util") pod "e4bef988-f09d-4f8a-b55b-20f1c194929a" (UID: "e4bef988-f09d-4f8a-b55b-20f1c194929a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:20.553715 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:20.553686 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:46:20.553715 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:20.553711 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4bef988-f09d-4f8a-b55b-20f1c194929a-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:46:20.553862 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:20.553721 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6d72\" (UniqueName: \"kubernetes.io/projected/e4bef988-f09d-4f8a-b55b-20f1c194929a-kube-api-access-l6d72\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:46:21.179482 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:21.179450 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" Jul 22 16:46:21.179482 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:21.179465 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxwb56" event={"ID":"e4bef988-f09d-4f8a-b55b-20f1c194929a","Type":"ContainerDied","Data":"b62f821ecd59482b0e967c73ab6b3cabd44bfff701d321b9b580585b03163014"} Jul 22 16:46:21.179710 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:21.179514 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b62f821ecd59482b0e967c73ab6b3cabd44bfff701d321b9b580585b03163014" Jul 22 16:46:21.180977 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:21.180953 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" event={"ID":"712bb46d-d73d-4c56-a3c1-509cd0572a99","Type":"ContainerStarted","Data":"629d04936fdb0cecfabc6b210047ac28e0ce8b4a308664d265d8cd339c5dfa7a"} Jul 22 16:46:21.206382 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:21.206333 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tgj97" podStartSLOduration=2.486915164 podStartE2EDuration="5.206320475s" totalImagesPullingTime="2.719405311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:46:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:46:21.205297878 +0000 UTC m=+368.115955050" watchObservedRunningTime="2026-07-22 16:46:21.206320475 +0000 UTC m=+368.116977647" Jul 22 16:46:24.902750 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.902716 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h5nkb"] Jul 22 16:46:24.903099 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.902990 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4bef988-f09d-4f8a-b55b-20f1c194929a" containerName="extract" Jul 22 16:46:24.903099 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.903001 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4bef988-f09d-4f8a-b55b-20f1c194929a" containerName="extract" Jul 22 16:46:24.903099 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.903010 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4bef988-f09d-4f8a-b55b-20f1c194929a" containerName="util" Jul 22 16:46:24.903099 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.903017 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4bef988-f09d-4f8a-b55b-20f1c194929a" containerName="util" Jul 22 16:46:24.903099 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.903031 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4bef988-f09d-4f8a-b55b-20f1c194929a" containerName="pull" Jul 22 16:46:24.903099 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.903036 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4bef988-f09d-4f8a-b55b-20f1c194929a" containerName="pull" Jul 22 16:46:24.903099 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.903084 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4bef988-f09d-4f8a-b55b-20f1c194929a" containerName="extract" Jul 22 16:46:24.908521 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.908500 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h5nkb" Jul 22 16:46:24.910915 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.910895 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nkkqh\"" Jul 22 16:46:24.933977 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.933956 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h5nkb"] Jul 22 16:46:24.986065 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.986036 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnv97\" (UniqueName: \"kubernetes.io/projected/1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622-kube-api-access-xnv97\") pod \"cert-manager-55b4d954d5-h5nkb\" (UID: \"1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622\") " pod="cert-manager/cert-manager-55b4d954d5-h5nkb" Jul 22 16:46:24.986163 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:24.986067 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622-bound-sa-token\") pod \"cert-manager-55b4d954d5-h5nkb\" (UID: \"1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622\") " pod="cert-manager/cert-manager-55b4d954d5-h5nkb" Jul 22 16:46:25.086804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:25.086775 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnv97\" (UniqueName: \"kubernetes.io/projected/1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622-kube-api-access-xnv97\") pod \"cert-manager-55b4d954d5-h5nkb\" (UID: \"1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622\") " pod="cert-manager/cert-manager-55b4d954d5-h5nkb" Jul 22 16:46:25.086804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:25.086807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622-bound-sa-token\") pod \"cert-manager-55b4d954d5-h5nkb\" (UID: \"1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622\") " pod="cert-manager/cert-manager-55b4d954d5-h5nkb" Jul 22 16:46:25.098648 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:25.098618 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnv97\" (UniqueName: \"kubernetes.io/projected/1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622-kube-api-access-xnv97\") pod \"cert-manager-55b4d954d5-h5nkb\" (UID: \"1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622\") " pod="cert-manager/cert-manager-55b4d954d5-h5nkb" Jul 22 16:46:25.098770 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:25.098743 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622-bound-sa-token\") pod \"cert-manager-55b4d954d5-h5nkb\" (UID: \"1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622\") " pod="cert-manager/cert-manager-55b4d954d5-h5nkb" Jul 22 16:46:25.216628 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:25.216541 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h5nkb" Jul 22 16:46:25.380476 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:25.380441 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h5nkb"] Jul 22 16:46:25.383714 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:46:25.383689 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e983b63_1b0c_4bfb_b7a7_7ab83bbd8622.slice/crio-8778a53eaa1ffd8fa99fc641958831f5aedaae13e08071113bf6cff1ae4042da WatchSource:0}: Error finding container 8778a53eaa1ffd8fa99fc641958831f5aedaae13e08071113bf6cff1ae4042da: Status 404 returned error can't find the container with id 8778a53eaa1ffd8fa99fc641958831f5aedaae13e08071113bf6cff1ae4042da Jul 22 16:46:26.204864 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:26.204833 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h5nkb" event={"ID":"1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622","Type":"ContainerStarted","Data":"5d1fdc11d3e68de9718b446fb9fa24482b7730252f0a3a2ea83d3b15a3b09153"} Jul 22 16:46:26.205217 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:26.204875 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h5nkb" event={"ID":"1e983b63-1b0c-4bfb-b7a7-7ab83bbd8622","Type":"ContainerStarted","Data":"8778a53eaa1ffd8fa99fc641958831f5aedaae13e08071113bf6cff1ae4042da"} Jul 22 16:46:26.255914 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:26.255863 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-h5nkb" podStartSLOduration=2.2558461100000002 podStartE2EDuration="2.25584611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:46:26.242677766 +0000 UTC m=+373.153334939" watchObservedRunningTime="2026-07-22 16:46:26.25584611 +0000 UTC m=+373.166503282" Jul 22 16:46:32.811986 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:32.811954 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m"] Jul 22 16:46:32.856852 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:32.856826 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m"] Jul 22 16:46:32.856988 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:32.856916 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:32.861612 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:32.861591 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 16:46:32.862734 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:32.862712 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 16:46:32.862829 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:32.862787 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhhz7\"" Jul 22 16:46:32.945580 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:32.945556 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl9wk\" (UniqueName: \"kubernetes.io/projected/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-kube-api-access-kl9wk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:32.945683 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:32.945628 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:32.945683 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:32.945647 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:33.045959 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:33.045935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:33.046041 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:33.045966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:33.046041 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:33.045988 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl9wk\" (UniqueName: \"kubernetes.io/projected/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-kube-api-access-kl9wk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:33.046330 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:33.046309 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:33.046410 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:33.046353 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:33.057629 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:33.057606 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl9wk\" (UniqueName: \"kubernetes.io/projected/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-kube-api-access-kl9wk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:33.165998 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:33.165946 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:33.344127 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:33.344092 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m"] Jul 22 16:46:33.347564 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:46:33.347536 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa2b0d4_d600_4f22_9f4f_e823994ca1d7.slice/crio-ae886642f0160e1be6f96300727ecf9080b760e9a7fc806bcd8c1b49e29c8e51 WatchSource:0}: Error finding container ae886642f0160e1be6f96300727ecf9080b760e9a7fc806bcd8c1b49e29c8e51: Status 404 returned error can't find the container with id ae886642f0160e1be6f96300727ecf9080b760e9a7fc806bcd8c1b49e29c8e51 Jul 22 16:46:34.242083 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:34.242041 2581 generic.go:356] "Generic (PLEG): container finished" podID="ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" containerID="4858701902de9c2cb95429e3800f45784fd45027eeba68e2b1613afa00cd1a5b" exitCode=0 Jul 22 16:46:34.242477 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:34.242082 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" event={"ID":"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7","Type":"ContainerDied","Data":"4858701902de9c2cb95429e3800f45784fd45027eeba68e2b1613afa00cd1a5b"} Jul 22 16:46:34.242477 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:34.242133 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" event={"ID":"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7","Type":"ContainerStarted","Data":"ae886642f0160e1be6f96300727ecf9080b760e9a7fc806bcd8c1b49e29c8e51"} Jul 22 16:46:37.259568 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:37.259537 2581 generic.go:356] "Generic (PLEG): container finished" podID="ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" containerID="cd63b969288fec0bc96cff05b3d58c520e93cda6d4e24bd7ee14d0c498341d81" exitCode=0 Jul 22 16:46:37.259929 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:37.259586 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" event={"ID":"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7","Type":"ContainerDied","Data":"cd63b969288fec0bc96cff05b3d58c520e93cda6d4e24bd7ee14d0c498341d81"} Jul 22 16:46:38.266512 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:38.266471 2581 generic.go:356] "Generic (PLEG): container finished" podID="ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" containerID="f12643b09af2d662b164ddc8e5489a71507d195e211e73790da64043e2c9cf43" exitCode=0 Jul 22 16:46:38.266897 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:38.266547 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" event={"ID":"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7","Type":"ContainerDied","Data":"f12643b09af2d662b164ddc8e5489a71507d195e211e73790da64043e2c9cf43"} Jul 22 16:46:39.395556 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:39.395531 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:39.491360 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:39.491332 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-util\") pod \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " Jul 22 16:46:39.491509 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:39.491423 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-bundle\") pod \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " Jul 22 16:46:39.491509 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:39.491463 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl9wk\" (UniqueName: \"kubernetes.io/projected/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-kube-api-access-kl9wk\") pod \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\" (UID: \"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7\") " Jul 22 16:46:39.492133 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:39.492106 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-bundle" (OuterVolumeSpecName: "bundle") pod "ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" (UID: "ffa2b0d4-d600-4f22-9f4f-e823994ca1d7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:39.493544 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:39.493519 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-kube-api-access-kl9wk" (OuterVolumeSpecName: "kube-api-access-kl9wk") pod "ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" (UID: "ffa2b0d4-d600-4f22-9f4f-e823994ca1d7"). InnerVolumeSpecName "kube-api-access-kl9wk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:46:39.496510 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:39.496490 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-util" (OuterVolumeSpecName: "util") pod "ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" (UID: "ffa2b0d4-d600-4f22-9f4f-e823994ca1d7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:46:39.592476 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:39.592418 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:46:39.592476 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:39.592439 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:46:39.592476 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:39.592452 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl9wk\" (UniqueName: \"kubernetes.io/projected/ffa2b0d4-d600-4f22-9f4f-e823994ca1d7-kube-api-access-kl9wk\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:46:40.277866 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:40.277824 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" event={"ID":"ffa2b0d4-d600-4f22-9f4f-e823994ca1d7","Type":"ContainerDied","Data":"ae886642f0160e1be6f96300727ecf9080b760e9a7fc806bcd8c1b49e29c8e51"} Jul 22 16:46:40.277866 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:40.277852 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlq5m" Jul 22 16:46:40.278058 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:40.277881 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae886642f0160e1be6f96300727ecf9080b760e9a7fc806bcd8c1b49e29c8e51" Jul 22 16:46:55.987643 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.987600 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch"] Jul 22 16:46:55.988152 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.988013 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" containerName="extract" Jul 22 16:46:55.988152 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.988033 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" containerName="extract" Jul 22 16:46:55.988152 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.988055 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" containerName="util" Jul 22 16:46:55.988152 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.988065 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" containerName="util" Jul 22 16:46:55.988152 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.988086 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" containerName="pull" Jul 22 16:46:55.988152 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.988094 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" containerName="pull" Jul 22 16:46:55.988486 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.988178 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffa2b0d4-d600-4f22-9f4f-e823994ca1d7" containerName="extract" Jul 22 16:46:55.992521 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.992499 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:55.996181 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.996159 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 16:46:55.996308 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.996261 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 16:46:55.996880 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.996864 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4qzcs\"" Jul 22 16:46:55.997021 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.997005 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 16:46:55.997107 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:55.997091 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 16:46:56.029467 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.029439 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch"] Jul 22 16:46:56.117637 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.117610 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddf1371f-2f06-4957-99b9-b7103af01681-tmp\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.117763 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.117641 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ddf1371f-2f06-4957-99b9-b7103af01681-manifests\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.117763 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.117667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ddf1371f-2f06-4957-99b9-b7103af01681-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.117833 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.117753 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzbxq\" (UniqueName: \"kubernetes.io/projected/ddf1371f-2f06-4957-99b9-b7103af01681-kube-api-access-zzbxq\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.117833 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.117789 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddf1371f-2f06-4957-99b9-b7103af01681-webhook-cert\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.218228 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.218199 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddf1371f-2f06-4957-99b9-b7103af01681-tmp\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.218343 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.218236 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ddf1371f-2f06-4957-99b9-b7103af01681-manifests\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.218435 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.218358 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ddf1371f-2f06-4957-99b9-b7103af01681-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.218497 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.218450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzbxq\" (UniqueName: \"kubernetes.io/projected/ddf1371f-2f06-4957-99b9-b7103af01681-kube-api-access-zzbxq\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.218497 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.218474 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddf1371f-2f06-4957-99b9-b7103af01681-webhook-cert\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.218675 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.218648 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ddf1371f-2f06-4957-99b9-b7103af01681-manifests\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.220519 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.220500 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddf1371f-2f06-4957-99b9-b7103af01681-tmp\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.220727 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.220710 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ddf1371f-2f06-4957-99b9-b7103af01681-webhook-cert\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.220881 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.220862 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ddf1371f-2f06-4957-99b9-b7103af01681-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.244614 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.244555 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzbxq\" (UniqueName: \"kubernetes.io/projected/ddf1371f-2f06-4957-99b9-b7103af01681-kube-api-access-zzbxq\") pod \"opendatahub-operator-controller-manager-55b7bf7664-vz8ch\" (UID: \"ddf1371f-2f06-4957-99b9-b7103af01681\") " pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.301978 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.301951 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:46:56.469304 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:56.469176 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch"] Jul 22 16:46:56.474439 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:46:56.474411 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf1371f_2f06_4957_99b9_b7103af01681.slice/crio-ab8ad44aab3d37c6784f2216e1ed25015ea78000327581fbb772404986ad13de WatchSource:0}: Error finding container ab8ad44aab3d37c6784f2216e1ed25015ea78000327581fbb772404986ad13de: Status 404 returned error can't find the container with id ab8ad44aab3d37c6784f2216e1ed25015ea78000327581fbb772404986ad13de Jul 22 16:46:57.354642 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:46:57.354594 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" event={"ID":"ddf1371f-2f06-4957-99b9-b7103af01681","Type":"ContainerStarted","Data":"ab8ad44aab3d37c6784f2216e1ed25015ea78000327581fbb772404986ad13de"} Jul 22 16:47:00.370726 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:00.370680 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" event={"ID":"ddf1371f-2f06-4957-99b9-b7103af01681","Type":"ContainerStarted","Data":"fdf62323012607467477b144dd0a2f2157f3cc4b406503b9ac044e60a00462f1"} Jul 22 16:47:01.379437 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:01.379399 2581 generic.go:356] "Generic (PLEG): container finished" podID="ddf1371f-2f06-4957-99b9-b7103af01681" containerID="fdf62323012607467477b144dd0a2f2157f3cc4b406503b9ac044e60a00462f1" exitCode=0 Jul 22 16:47:01.379819 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:01.379462 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" event={"ID":"ddf1371f-2f06-4957-99b9-b7103af01681","Type":"ContainerDied","Data":"fdf62323012607467477b144dd0a2f2157f3cc4b406503b9ac044e60a00462f1"} Jul 22 16:47:02.384957 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:02.384923 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" event={"ID":"ddf1371f-2f06-4957-99b9-b7103af01681","Type":"ContainerStarted","Data":"3e4d256fe6ab340b460988433e5f3ced1ea4e8811b9051e7d41750f1899133fa"} Jul 22 16:47:02.385344 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:02.385030 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:47:02.447782 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:02.447741 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" podStartSLOduration=3.098036002 podStartE2EDuration="7.447726912s" totalImagesPullingTime="3.314068515s" totalInitContainerRuntime="1.035622395s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:46:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:47:02.445078664 +0000 UTC m=+409.355735836" watchObservedRunningTime="2026-07-22 16:47:02.447726912 +0000 UTC m=+409.358384085" Jul 22 16:47:13.392224 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:13.392186 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55b7bf7664-vz8ch" Jul 22 16:47:17.916177 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:17.916144 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-c467c957f-cmt89"] Jul 22 16:47:17.925114 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:17.925084 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:17.935021 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:17.935003 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 22 16:47:17.935111 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:17.935081 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jgl2w\"" Jul 22 16:47:17.950014 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:17.949995 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 22 16:47:18.053260 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.053236 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c467c957f-cmt89"] Jul 22 16:47:18.083617 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.083592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fd3d84-4847-4c3c-b815-e99e522f8225-tls-certs\") pod \"kube-auth-proxy-c467c957f-cmt89\" (UID: \"e8fd3d84-4847-4c3c-b815-e99e522f8225\") " pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:18.083748 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.083644 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24rgc\" (UniqueName: \"kubernetes.io/projected/e8fd3d84-4847-4c3c-b815-e99e522f8225-kube-api-access-24rgc\") pod \"kube-auth-proxy-c467c957f-cmt89\" (UID: \"e8fd3d84-4847-4c3c-b815-e99e522f8225\") " pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:18.083748 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.083692 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8fd3d84-4847-4c3c-b815-e99e522f8225-tmp\") pod \"kube-auth-proxy-c467c957f-cmt89\" (UID: \"e8fd3d84-4847-4c3c-b815-e99e522f8225\") " pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:18.134211 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.134187 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bnw5n"] Jul 22 16:47:18.137422 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.137407 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:18.141178 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.141154 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 22 16:47:18.141288 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.141188 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qqzf4\"" Jul 22 16:47:18.184628 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.184579 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fd3d84-4847-4c3c-b815-e99e522f8225-tls-certs\") pod \"kube-auth-proxy-c467c957f-cmt89\" (UID: \"e8fd3d84-4847-4c3c-b815-e99e522f8225\") " pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:18.184726 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.184631 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24rgc\" (UniqueName: \"kubernetes.io/projected/e8fd3d84-4847-4c3c-b815-e99e522f8225-kube-api-access-24rgc\") pod \"kube-auth-proxy-c467c957f-cmt89\" (UID: \"e8fd3d84-4847-4c3c-b815-e99e522f8225\") " pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:18.184726 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.184655 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8fd3d84-4847-4c3c-b815-e99e522f8225-tmp\") pod \"kube-auth-proxy-c467c957f-cmt89\" (UID: \"e8fd3d84-4847-4c3c-b815-e99e522f8225\") " pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:18.186803 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.186782 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8fd3d84-4847-4c3c-b815-e99e522f8225-tmp\") pod \"kube-auth-proxy-c467c957f-cmt89\" (UID: \"e8fd3d84-4847-4c3c-b815-e99e522f8225\") " pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:18.186967 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.186949 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8fd3d84-4847-4c3c-b815-e99e522f8225-tls-certs\") pod \"kube-auth-proxy-c467c957f-cmt89\" (UID: \"e8fd3d84-4847-4c3c-b815-e99e522f8225\") " pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:18.210747 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.210724 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bnw5n"] Jul 22 16:47:18.234325 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.234303 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24rgc\" (UniqueName: \"kubernetes.io/projected/e8fd3d84-4847-4c3c-b815-e99e522f8225-kube-api-access-24rgc\") pod \"kube-auth-proxy-c467c957f-cmt89\" (UID: \"e8fd3d84-4847-4c3c-b815-e99e522f8225\") " pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:18.285491 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.285468 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0709664-86d3-4650-8693-d835271dc46d-cert\") pod \"odh-model-controller-7998c67dc4-bnw5n\" (UID: \"f0709664-86d3-4650-8693-d835271dc46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:18.285579 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.285512 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc7wj\" (UniqueName: \"kubernetes.io/projected/f0709664-86d3-4650-8693-d835271dc46d-kube-api-access-sc7wj\") pod \"odh-model-controller-7998c67dc4-bnw5n\" (UID: \"f0709664-86d3-4650-8693-d835271dc46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:18.386436 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.386414 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0709664-86d3-4650-8693-d835271dc46d-cert\") pod \"odh-model-controller-7998c67dc4-bnw5n\" (UID: \"f0709664-86d3-4650-8693-d835271dc46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:18.386580 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.386463 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc7wj\" (UniqueName: \"kubernetes.io/projected/f0709664-86d3-4650-8693-d835271dc46d-kube-api-access-sc7wj\") pod \"odh-model-controller-7998c67dc4-bnw5n\" (UID: \"f0709664-86d3-4650-8693-d835271dc46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:18.386580 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:47:18.386569 2581 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 22 16:47:18.386661 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:47:18.386636 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0709664-86d3-4650-8693-d835271dc46d-cert podName:f0709664-86d3-4650-8693-d835271dc46d nodeName:}" failed. No retries permitted until 2026-07-22 16:47:18.886618124 +0000 UTC m=+425.797275284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f0709664-86d3-4650-8693-d835271dc46d-cert") pod "odh-model-controller-7998c67dc4-bnw5n" (UID: "f0709664-86d3-4650-8693-d835271dc46d") : secret "odh-model-controller-webhook-cert" not found Jul 22 16:47:18.424689 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.424659 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc7wj\" (UniqueName: \"kubernetes.io/projected/f0709664-86d3-4650-8693-d835271dc46d-kube-api-access-sc7wj\") pod \"odh-model-controller-7998c67dc4-bnw5n\" (UID: \"f0709664-86d3-4650-8693-d835271dc46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:18.534268 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.534247 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" Jul 22 16:47:18.800843 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.800819 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c467c957f-cmt89"] Jul 22 16:47:18.802126 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:47:18.802103 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8fd3d84_4847_4c3c_b815_e99e522f8225.slice/crio-b7206922a9ddc50b52bd591c6ba70bc97c431c03299c318ae01d08bdf49ba6b6 WatchSource:0}: Error finding container b7206922a9ddc50b52bd591c6ba70bc97c431c03299c318ae01d08bdf49ba6b6: Status 404 returned error can't find the container with id b7206922a9ddc50b52bd591c6ba70bc97c431c03299c318ae01d08bdf49ba6b6 Jul 22 16:47:18.890047 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.890022 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0709664-86d3-4650-8693-d835271dc46d-cert\") pod \"odh-model-controller-7998c67dc4-bnw5n\" (UID: \"f0709664-86d3-4650-8693-d835271dc46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:18.892421 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:18.892398 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0709664-86d3-4650-8693-d835271dc46d-cert\") pod \"odh-model-controller-7998c67dc4-bnw5n\" (UID: \"f0709664-86d3-4650-8693-d835271dc46d\") " pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:19.047036 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:19.047011 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:19.304119 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:19.304056 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bnw5n"] Jul 22 16:47:19.347578 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:47:19.347551 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0709664_86d3_4650_8693_d835271dc46d.slice/crio-9cb914277ff401a09932eb3a6c48a54b3a0a99a5bcf8be65652beefed2ac2c47 WatchSource:0}: Error finding container 9cb914277ff401a09932eb3a6c48a54b3a0a99a5bcf8be65652beefed2ac2c47: Status 404 returned error can't find the container with id 9cb914277ff401a09932eb3a6c48a54b3a0a99a5bcf8be65652beefed2ac2c47 Jul 22 16:47:19.463944 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:19.463902 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" event={"ID":"e8fd3d84-4847-4c3c-b815-e99e522f8225","Type":"ContainerStarted","Data":"b7206922a9ddc50b52bd591c6ba70bc97c431c03299c318ae01d08bdf49ba6b6"} Jul 22 16:47:19.465424 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:19.465397 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" event={"ID":"f0709664-86d3-4650-8693-d835271dc46d","Type":"ContainerStarted","Data":"9cb914277ff401a09932eb3a6c48a54b3a0a99a5bcf8be65652beefed2ac2c47"} Jul 22 16:47:20.541280 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.541246 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4xpnv"] Jul 22 16:47:20.544626 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.544604 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" Jul 22 16:47:20.658155 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.658077 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4xpnv"] Jul 22 16:47:20.707879 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.707843 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dkp5\" (UniqueName: \"kubernetes.io/projected/01c0b29f-fa6c-4777-96f6-fb7d39a01f23-kube-api-access-7dkp5\") pod \"model-serving-api-54d89bd5fc-4xpnv\" (UID: \"01c0b29f-fa6c-4777-96f6-fb7d39a01f23\") " pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" Jul 22 16:47:20.708069 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.707989 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c0b29f-fa6c-4777-96f6-fb7d39a01f23-tls-certs\") pod \"model-serving-api-54d89bd5fc-4xpnv\" (UID: \"01c0b29f-fa6c-4777-96f6-fb7d39a01f23\") " pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" Jul 22 16:47:20.725519 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.725302 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-hctcq\"" Jul 22 16:47:20.726665 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.726644 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 22 16:47:20.808682 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.808595 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c0b29f-fa6c-4777-96f6-fb7d39a01f23-tls-certs\") pod \"model-serving-api-54d89bd5fc-4xpnv\" (UID: \"01c0b29f-fa6c-4777-96f6-fb7d39a01f23\") " pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" Jul 22 16:47:20.808682 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.808678 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dkp5\" (UniqueName: \"kubernetes.io/projected/01c0b29f-fa6c-4777-96f6-fb7d39a01f23-kube-api-access-7dkp5\") pod \"model-serving-api-54d89bd5fc-4xpnv\" (UID: \"01c0b29f-fa6c-4777-96f6-fb7d39a01f23\") " pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" Jul 22 16:47:20.812630 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.812581 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c0b29f-fa6c-4777-96f6-fb7d39a01f23-tls-certs\") pod \"model-serving-api-54d89bd5fc-4xpnv\" (UID: \"01c0b29f-fa6c-4777-96f6-fb7d39a01f23\") " pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" Jul 22 16:47:20.899129 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:20.899098 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dkp5\" (UniqueName: \"kubernetes.io/projected/01c0b29f-fa6c-4777-96f6-fb7d39a01f23-kube-api-access-7dkp5\") pod \"model-serving-api-54d89bd5fc-4xpnv\" (UID: \"01c0b29f-fa6c-4777-96f6-fb7d39a01f23\") " pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" Jul 22 16:47:21.158719 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:21.158630 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" Jul 22 16:47:22.668210 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:22.668175 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4xpnv"] Jul 22 16:47:22.671736 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:47:22.671704 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c0b29f_fa6c_4777_96f6_fb7d39a01f23.slice/crio-5aaa08c870f7c7abdb7f2fd4f0454b4c62c2d0f5c81eecfdfae798cca1a292f9 WatchSource:0}: Error finding container 5aaa08c870f7c7abdb7f2fd4f0454b4c62c2d0f5c81eecfdfae798cca1a292f9: Status 404 returned error can't find the container with id 5aaa08c870f7c7abdb7f2fd4f0454b4c62c2d0f5c81eecfdfae798cca1a292f9 Jul 22 16:47:23.491804 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:23.491746 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" event={"ID":"e8fd3d84-4847-4c3c-b815-e99e522f8225","Type":"ContainerStarted","Data":"14aa8e38b4df6a65c275adff82140d3d5b0ff47053c3eea621762994a256b333"} Jul 22 16:47:23.493739 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:23.493706 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" event={"ID":"f0709664-86d3-4650-8693-d835271dc46d","Type":"ContainerStarted","Data":"cc44c4a8980c7f0a8ab153a9c8626cdc21c0b1ab91528b1a2e03a5c7d90497b7"} Jul 22 16:47:23.493860 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:23.493777 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:23.495067 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:23.495041 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" event={"ID":"01c0b29f-fa6c-4777-96f6-fb7d39a01f23","Type":"ContainerStarted","Data":"5aaa08c870f7c7abdb7f2fd4f0454b4c62c2d0f5c81eecfdfae798cca1a292f9"} Jul 22 16:47:23.548325 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:23.548275 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-c467c957f-cmt89" podStartSLOduration=2.8267912920000002 podStartE2EDuration="6.548258057s" totalImagesPullingTime="3.721466765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:47:23.547132012 +0000 UTC m=+430.457789185" watchObservedRunningTime="2026-07-22 16:47:23.548258057 +0000 UTC m=+430.458915250" Jul 22 16:47:23.595268 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:23.595218 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" podStartSLOduration=2.370181453 podStartE2EDuration="5.595195806s" totalImagesPullingTime="3.225014353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:47:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:47:23.590479403 +0000 UTC m=+430.501136575" watchObservedRunningTime="2026-07-22 16:47:23.595195806 +0000 UTC m=+430.505852979" Jul 22 16:47:24.500974 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:24.500937 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" event={"ID":"01c0b29f-fa6c-4777-96f6-fb7d39a01f23","Type":"ContainerStarted","Data":"4872d95714fee9b918193f895cb6450a1b02d771600bb97bad6fd3c7cdbea357"} Jul 22 16:47:24.543234 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:24.543184 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" podStartSLOduration=3.158262599 podStartE2EDuration="4.543168259s" totalImagesPullingTime="1.38490566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:47:24.535637114 +0000 UTC m=+431.446294287" watchObservedRunningTime="2026-07-22 16:47:24.543168259 +0000 UTC m=+431.453825430" Jul 22 16:47:25.506095 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:25.506056 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" Jul 22 16:47:27.012188 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.012151 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w"] Jul 22 16:47:27.015554 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.015536 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.018886 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.018853 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 16:47:27.019770 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.019747 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 16:47:27.019885 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.019750 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 16:47:27.019885 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.019752 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:47:27.019885 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.019805 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 16:47:27.019885 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.019773 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wvk7r\"" Jul 22 16:47:27.041195 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.041177 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w"] Jul 22 16:47:27.164910 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.164879 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c12eb5d9-8d51-40b4-914c-3e95eacf2379-cert\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.165054 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.164921 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmw95\" (UniqueName: \"kubernetes.io/projected/c12eb5d9-8d51-40b4-914c-3e95eacf2379-kube-api-access-tmw95\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.165054 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.164978 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c12eb5d9-8d51-40b4-914c-3e95eacf2379-manager-config\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.165054 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.165030 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c12eb5d9-8d51-40b4-914c-3e95eacf2379-metrics-cert\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.266255 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.266178 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c12eb5d9-8d51-40b4-914c-3e95eacf2379-metrics-cert\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.266255 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.266254 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c12eb5d9-8d51-40b4-914c-3e95eacf2379-cert\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.266490 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.266274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmw95\" (UniqueName: \"kubernetes.io/projected/c12eb5d9-8d51-40b4-914c-3e95eacf2379-kube-api-access-tmw95\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.266490 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.266293 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c12eb5d9-8d51-40b4-914c-3e95eacf2379-manager-config\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.266996 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.266977 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c12eb5d9-8d51-40b4-914c-3e95eacf2379-manager-config\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.268655 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.268635 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c12eb5d9-8d51-40b4-914c-3e95eacf2379-metrics-cert\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.268803 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.268782 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c12eb5d9-8d51-40b4-914c-3e95eacf2379-cert\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.293030 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.293007 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmw95\" (UniqueName: \"kubernetes.io/projected/c12eb5d9-8d51-40b4-914c-3e95eacf2379-kube-api-access-tmw95\") pod \"lws-controller-manager-57844c55d9-5gb2w\" (UID: \"c12eb5d9-8d51-40b4-914c-3e95eacf2379\") " pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.324919 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.324888 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:27.498759 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.498722 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w"] Jul 22 16:47:27.501646 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:47:27.501621 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc12eb5d9_8d51_40b4_914c_3e95eacf2379.slice/crio-e72f7a103227a8add89325a916af35ac465337ba06e70d5aaf931ef7df1a575a WatchSource:0}: Error finding container e72f7a103227a8add89325a916af35ac465337ba06e70d5aaf931ef7df1a575a: Status 404 returned error can't find the container with id e72f7a103227a8add89325a916af35ac465337ba06e70d5aaf931ef7df1a575a Jul 22 16:47:27.516540 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:27.516481 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" event={"ID":"c12eb5d9-8d51-40b4-914c-3e95eacf2379","Type":"ContainerStarted","Data":"e72f7a103227a8add89325a916af35ac465337ba06e70d5aaf931ef7df1a575a"} Jul 22 16:47:29.527932 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:29.527900 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" event={"ID":"c12eb5d9-8d51-40b4-914c-3e95eacf2379","Type":"ContainerStarted","Data":"6e3e184a8e6c2e8b9e434407e4af3c18445621c55a2fcf5cd306e89e43138bb4"} Jul 22 16:47:29.528307 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:29.528029 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:47:29.561385 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:29.561325 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" podStartSLOduration=1.921673657 podStartE2EDuration="3.561310336s" totalImagesPullingTime="1.639636679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:47:29.55979587 +0000 UTC m=+436.470453042" watchObservedRunningTime="2026-07-22 16:47:29.561310336 +0000 UTC m=+436.471967507" Jul 22 16:47:34.503047 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:34.503016 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bnw5n" Jul 22 16:47:36.515119 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:36.515038 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4xpnv" Jul 22 16:47:40.535148 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:47:40.535116 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57844c55d9-5gb2w" Jul 22 16:48:13.901078 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:13.901043 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fc48fb58d-fp4xd"] Jul 22 16:48:13.907038 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:13.907010 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:13.916077 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:13.916051 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc48fb58d-fp4xd"] Jul 22 16:48:14.007089 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.007057 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-oauth-serving-cert\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.007229 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.007105 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/889fad7f-aa13-4bef-9200-1e3a9f37f685-console-serving-cert\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.007229 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.007124 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-service-ca\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.007229 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.007214 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-console-config\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.007378 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.007241 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-trusted-ca-bundle\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.007378 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.007258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rnr6\" (UniqueName: \"kubernetes.io/projected/889fad7f-aa13-4bef-9200-1e3a9f37f685-kube-api-access-4rnr6\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.007378 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.007279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/889fad7f-aa13-4bef-9200-1e3a9f37f685-console-oauth-config\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.108468 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.108440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-console-config\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.108468 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.108469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-trusted-ca-bundle\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.108671 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.108486 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rnr6\" (UniqueName: \"kubernetes.io/projected/889fad7f-aa13-4bef-9200-1e3a9f37f685-kube-api-access-4rnr6\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.108671 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.108507 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/889fad7f-aa13-4bef-9200-1e3a9f37f685-console-oauth-config\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.108671 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.108530 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-oauth-serving-cert\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.108671 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.108582 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/889fad7f-aa13-4bef-9200-1e3a9f37f685-console-serving-cert\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.108671 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.108597 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-service-ca\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.109392 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.109342 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-oauth-serving-cert\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.109503 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.109406 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-trusted-ca-bundle\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.109503 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.109443 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-service-ca\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.109503 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.109454 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/889fad7f-aa13-4bef-9200-1e3a9f37f685-console-config\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.111186 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.111166 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/889fad7f-aa13-4bef-9200-1e3a9f37f685-console-oauth-config\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.111244 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.111176 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/889fad7f-aa13-4bef-9200-1e3a9f37f685-console-serving-cert\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.119985 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.119965 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rnr6\" (UniqueName: \"kubernetes.io/projected/889fad7f-aa13-4bef-9200-1e3a9f37f685-kube-api-access-4rnr6\") pod \"console-6fc48fb58d-fp4xd\" (UID: \"889fad7f-aa13-4bef-9200-1e3a9f37f685\") " pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.218159 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.218107 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:14.353159 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.353130 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc48fb58d-fp4xd"] Jul 22 16:48:14.354995 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:48:14.354972 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889fad7f_aa13_4bef_9200_1e3a9f37f685.slice/crio-e0fdc316dab066e938b727b8b5f9cbed0e72b6203f750eda24c24ee711507bb8 WatchSource:0}: Error finding container e0fdc316dab066e938b727b8b5f9cbed0e72b6203f750eda24c24ee711507bb8: Status 404 returned error can't find the container with id e0fdc316dab066e938b727b8b5f9cbed0e72b6203f750eda24c24ee711507bb8 Jul 22 16:48:14.733074 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.733037 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc48fb58d-fp4xd" event={"ID":"889fad7f-aa13-4bef-9200-1e3a9f37f685","Type":"ContainerStarted","Data":"efb9a76c2d642c529182fa9c453a731022bf17d339a7c34a21ae742c5ae66a9e"} Jul 22 16:48:14.733261 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.733086 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc48fb58d-fp4xd" event={"ID":"889fad7f-aa13-4bef-9200-1e3a9f37f685","Type":"ContainerStarted","Data":"e0fdc316dab066e938b727b8b5f9cbed0e72b6203f750eda24c24ee711507bb8"} Jul 22 16:48:14.753159 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:14.753111 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fc48fb58d-fp4xd" podStartSLOduration=1.7530956739999999 podStartE2EDuration="1.753095674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:48:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:48:14.752629703 +0000 UTC m=+481.663286904" watchObservedRunningTime="2026-07-22 16:48:14.753095674 +0000 UTC m=+481.663752912" Jul 22 16:48:24.218612 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:24.218574 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:24.219057 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:24.218641 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:24.223234 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:24.223210 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:24.783868 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:24.783839 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fc48fb58d-fp4xd" Jul 22 16:48:24.841528 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:24.841498 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8679dbbccf-84r4r"] Jul 22 16:48:34.049254 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.049222 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td"] Jul 22 16:48:34.053760 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.053743 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.056107 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.056081 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 16:48:34.056806 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.056792 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 16:48:34.056887 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.056796 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bfjkn\"" Jul 22 16:48:34.060553 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.060529 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td"] Jul 22 16:48:34.165997 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.165972 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzvm9\" (UniqueName: \"kubernetes.io/projected/d4b0b269-0edf-4d86-be61-d2a6c6b75701-kube-api-access-dzvm9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.166099 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.166024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.166099 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.166060 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.267119 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.267092 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.267215 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.267159 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.267215 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.267204 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzvm9\" (UniqueName: \"kubernetes.io/projected/d4b0b269-0edf-4d86-be61-d2a6c6b75701-kube-api-access-dzvm9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.267496 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.267480 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.267541 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.267503 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.275683 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.275664 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzvm9\" (UniqueName: \"kubernetes.io/projected/d4b0b269-0edf-4d86-be61-d2a6c6b75701-kube-api-access-dzvm9\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.363621 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.363565 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:34.492416 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.492393 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td"] Jul 22 16:48:34.494194 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:48:34.494169 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b0b269_0edf_4d86_be61_d2a6c6b75701.slice/crio-86ec436a24a1ce5d12f16b13a494744502a3d3daa30daa82d9a0f4bf659349d8 WatchSource:0}: Error finding container 86ec436a24a1ce5d12f16b13a494744502a3d3daa30daa82d9a0f4bf659349d8: Status 404 returned error can't find the container with id 86ec436a24a1ce5d12f16b13a494744502a3d3daa30daa82d9a0f4bf659349d8 Jul 22 16:48:34.664691 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.664638 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf"] Jul 22 16:48:34.668013 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.667997 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:34.676625 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.676603 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf"] Jul 22 16:48:34.770430 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.770402 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:34.770551 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.770483 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvdcw\" (UniqueName: \"kubernetes.io/projected/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-kube-api-access-vvdcw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:34.770551 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.770511 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:34.825156 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.825125 2581 generic.go:356] "Generic (PLEG): container finished" podID="d4b0b269-0edf-4d86-be61-d2a6c6b75701" containerID="98cb983d98ea26a87dd581ade087b48df68c8f8fc0405a6cc27db695fea778cc" exitCode=0 Jul 22 16:48:34.825275 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.825194 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" event={"ID":"d4b0b269-0edf-4d86-be61-d2a6c6b75701","Type":"ContainerDied","Data":"98cb983d98ea26a87dd581ade087b48df68c8f8fc0405a6cc27db695fea778cc"} Jul 22 16:48:34.825275 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.825251 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" event={"ID":"d4b0b269-0edf-4d86-be61-d2a6c6b75701","Type":"ContainerStarted","Data":"86ec436a24a1ce5d12f16b13a494744502a3d3daa30daa82d9a0f4bf659349d8"} Jul 22 16:48:34.871463 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.871433 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:34.871564 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.871522 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvdcw\" (UniqueName: \"kubernetes.io/projected/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-kube-api-access-vvdcw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:34.871621 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.871563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:34.871830 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.871807 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:34.871874 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.871858 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:34.880035 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.880018 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvdcw\" (UniqueName: \"kubernetes.io/projected/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-kube-api-access-vvdcw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:34.976886 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:34.976867 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:35.108321 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.108296 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf"] Jul 22 16:48:35.110206 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:48:35.110177 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dacb29f_cf97_4dc0_bd83_7a619c3f6085.slice/crio-ac4b99daa72444efb57ae33201fefb15bd0a36589f96ad6b4c449d78c9f99b97 WatchSource:0}: Error finding container ac4b99daa72444efb57ae33201fefb15bd0a36589f96ad6b4c449d78c9f99b97: Status 404 returned error can't find the container with id ac4b99daa72444efb57ae33201fefb15bd0a36589f96ad6b4c449d78c9f99b97 Jul 22 16:48:35.256991 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.256930 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj"] Jul 22 16:48:35.260326 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.260310 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.271304 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.271280 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj"] Jul 22 16:48:35.376972 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.376948 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28d9l\" (UniqueName: \"kubernetes.io/projected/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-kube-api-access-28d9l\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.377078 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.376987 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.377078 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.377014 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.477779 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.477751 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.477924 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.477797 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.477924 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.477882 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28d9l\" (UniqueName: \"kubernetes.io/projected/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-kube-api-access-28d9l\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.478193 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.478154 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.478193 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.478181 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.486718 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.486697 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28d9l\" (UniqueName: \"kubernetes.io/projected/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-kube-api-access-28d9l\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.569118 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.569094 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:35.657438 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.657407 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s"] Jul 22 16:48:35.661664 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.661638 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:35.669661 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.669640 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s"] Jul 22 16:48:35.704586 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.704564 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj"] Jul 22 16:48:35.706650 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:48:35.706627 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f0ba89f_d2fe_4e82_af1c_58d794054c9c.slice/crio-cf66b8fe9d4eab130f5ca94a30e57652986d49250600056d14b982e8665bcc78 WatchSource:0}: Error finding container cf66b8fe9d4eab130f5ca94a30e57652986d49250600056d14b982e8665bcc78: Status 404 returned error can't find the container with id cf66b8fe9d4eab130f5ca94a30e57652986d49250600056d14b982e8665bcc78 Jul 22 16:48:35.781274 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.781245 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vwgl\" (UniqueName: \"kubernetes.io/projected/1cc39293-34d0-408d-910f-79c8f4a3295d-kube-api-access-9vwgl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:35.781394 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.781291 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:35.781394 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.781319 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:35.831292 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.831267 2581 generic.go:356] "Generic (PLEG): container finished" podID="2f0ba89f-d2fe-4e82-af1c-58d794054c9c" containerID="04688a6e99f6c315dfe835729802a1de51e3fd4b55305c84594f46ce306d5f2b" exitCode=0 Jul 22 16:48:35.831423 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.831343 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" event={"ID":"2f0ba89f-d2fe-4e82-af1c-58d794054c9c","Type":"ContainerDied","Data":"04688a6e99f6c315dfe835729802a1de51e3fd4b55305c84594f46ce306d5f2b"} Jul 22 16:48:35.831423 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.831413 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" event={"ID":"2f0ba89f-d2fe-4e82-af1c-58d794054c9c","Type":"ContainerStarted","Data":"cf66b8fe9d4eab130f5ca94a30e57652986d49250600056d14b982e8665bcc78"} Jul 22 16:48:35.833193 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.833169 2581 generic.go:356] "Generic (PLEG): container finished" podID="d4b0b269-0edf-4d86-be61-d2a6c6b75701" containerID="990bff40f8a8b3e4f99db5d28cde4f260f00c45056f4e94b1d4af3456c759d68" exitCode=0 Jul 22 16:48:35.833433 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.833244 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" event={"ID":"d4b0b269-0edf-4d86-be61-d2a6c6b75701","Type":"ContainerDied","Data":"990bff40f8a8b3e4f99db5d28cde4f260f00c45056f4e94b1d4af3456c759d68"} Jul 22 16:48:35.834828 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.834797 2581 generic.go:356] "Generic (PLEG): container finished" podID="8dacb29f-cf97-4dc0-bd83-7a619c3f6085" containerID="545540bc0db5e86e7d207cd56368629a67a7a1120dffd08dc6e4ba6d7211849c" exitCode=0 Jul 22 16:48:35.834903 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.834855 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" event={"ID":"8dacb29f-cf97-4dc0-bd83-7a619c3f6085","Type":"ContainerDied","Data":"545540bc0db5e86e7d207cd56368629a67a7a1120dffd08dc6e4ba6d7211849c"} Jul 22 16:48:35.834967 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.834901 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" event={"ID":"8dacb29f-cf97-4dc0-bd83-7a619c3f6085","Type":"ContainerStarted","Data":"ac4b99daa72444efb57ae33201fefb15bd0a36589f96ad6b4c449d78c9f99b97"} Jul 22 16:48:35.882216 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.882189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:35.882295 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.882220 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:35.882342 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.882294 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vwgl\" (UniqueName: \"kubernetes.io/projected/1cc39293-34d0-408d-910f-79c8f4a3295d-kube-api-access-9vwgl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:35.882541 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.882520 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:35.882581 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.882553 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:35.892401 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.892383 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vwgl\" (UniqueName: \"kubernetes.io/projected/1cc39293-34d0-408d-910f-79c8f4a3295d-kube-api-access-9vwgl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:35.991492 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:35.991468 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:36.119118 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:36.119097 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s"] Jul 22 16:48:36.120950 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:48:36.120927 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc39293_34d0_408d_910f_79c8f4a3295d.slice/crio-0d769833080e88c46933f27cd5dc246be67ad74d4f31462c6f9c46c1bac59efb WatchSource:0}: Error finding container 0d769833080e88c46933f27cd5dc246be67ad74d4f31462c6f9c46c1bac59efb: Status 404 returned error can't find the container with id 0d769833080e88c46933f27cd5dc246be67ad74d4f31462c6f9c46c1bac59efb Jul 22 16:48:36.841138 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:36.841110 2581 generic.go:356] "Generic (PLEG): container finished" podID="2f0ba89f-d2fe-4e82-af1c-58d794054c9c" containerID="3d8e76ecfb262bc3da561f74f47eef4da612918a2cf70af4c53ecaf7e3f31b6f" exitCode=0 Jul 22 16:48:36.841243 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:36.841191 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" event={"ID":"2f0ba89f-d2fe-4e82-af1c-58d794054c9c","Type":"ContainerDied","Data":"3d8e76ecfb262bc3da561f74f47eef4da612918a2cf70af4c53ecaf7e3f31b6f"} Jul 22 16:48:36.842753 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:36.842732 2581 generic.go:356] "Generic (PLEG): container finished" podID="1cc39293-34d0-408d-910f-79c8f4a3295d" containerID="31eb2865e6e78ec85bc127b531d0778586c0c95f9e8e2d01ad00544d83a45ac9" exitCode=0 Jul 22 16:48:36.842839 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:36.842799 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" event={"ID":"1cc39293-34d0-408d-910f-79c8f4a3295d","Type":"ContainerDied","Data":"31eb2865e6e78ec85bc127b531d0778586c0c95f9e8e2d01ad00544d83a45ac9"} Jul 22 16:48:36.842895 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:36.842848 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" event={"ID":"1cc39293-34d0-408d-910f-79c8f4a3295d","Type":"ContainerStarted","Data":"0d769833080e88c46933f27cd5dc246be67ad74d4f31462c6f9c46c1bac59efb"} Jul 22 16:48:36.845788 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:36.845717 2581 generic.go:356] "Generic (PLEG): container finished" podID="d4b0b269-0edf-4d86-be61-d2a6c6b75701" containerID="99a4e7c00b4cca4a855ad59c1e47a05f37afaae242e67275362082bf8f5e5011" exitCode=0 Jul 22 16:48:36.845894 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:36.845872 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" event={"ID":"d4b0b269-0edf-4d86-be61-d2a6c6b75701","Type":"ContainerDied","Data":"99a4e7c00b4cca4a855ad59c1e47a05f37afaae242e67275362082bf8f5e5011"} Jul 22 16:48:36.852521 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:36.852501 2581 generic.go:356] "Generic (PLEG): container finished" podID="8dacb29f-cf97-4dc0-bd83-7a619c3f6085" containerID="1b7189d823c6a93b3ee0ec8cb0e85ad911e9b86c06568765019657de29d1cab1" exitCode=0 Jul 22 16:48:36.852629 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:36.852543 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" event={"ID":"8dacb29f-cf97-4dc0-bd83-7a619c3f6085","Type":"ContainerDied","Data":"1b7189d823c6a93b3ee0ec8cb0e85ad911e9b86c06568765019657de29d1cab1"} Jul 22 16:48:37.860951 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:37.860920 2581 generic.go:356] "Generic (PLEG): container finished" podID="2f0ba89f-d2fe-4e82-af1c-58d794054c9c" containerID="e653e0675345d28bbb0c9e8e9c3e9e8685fbd34af48f2272f47885bbb0a99ea6" exitCode=0 Jul 22 16:48:37.861292 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:37.861010 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" event={"ID":"2f0ba89f-d2fe-4e82-af1c-58d794054c9c","Type":"ContainerDied","Data":"e653e0675345d28bbb0c9e8e9c3e9e8685fbd34af48f2272f47885bbb0a99ea6"} Jul 22 16:48:37.862711 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:37.862686 2581 generic.go:356] "Generic (PLEG): container finished" podID="1cc39293-34d0-408d-910f-79c8f4a3295d" containerID="6beb83c38a292178ef7c602cc0f1ae79ac9fedd3eb3bb0245b1cfb96cdaa41d7" exitCode=0 Jul 22 16:48:37.862805 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:37.862716 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" event={"ID":"1cc39293-34d0-408d-910f-79c8f4a3295d","Type":"ContainerDied","Data":"6beb83c38a292178ef7c602cc0f1ae79ac9fedd3eb3bb0245b1cfb96cdaa41d7"} Jul 22 16:48:37.865129 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:37.865108 2581 generic.go:356] "Generic (PLEG): container finished" podID="8dacb29f-cf97-4dc0-bd83-7a619c3f6085" containerID="cb847c9692c3fb8f49eea14060553fe565f3768429946ce14c23eb7a1ab63445" exitCode=0 Jul 22 16:48:37.865210 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:37.865172 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" event={"ID":"8dacb29f-cf97-4dc0-bd83-7a619c3f6085","Type":"ContainerDied","Data":"cb847c9692c3fb8f49eea14060553fe565f3768429946ce14c23eb7a1ab63445"} Jul 22 16:48:37.995961 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:37.995938 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:38.102233 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.102207 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzvm9\" (UniqueName: \"kubernetes.io/projected/d4b0b269-0edf-4d86-be61-d2a6c6b75701-kube-api-access-dzvm9\") pod \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " Jul 22 16:48:38.102339 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.102253 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-util\") pod \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " Jul 22 16:48:38.102339 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.102333 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-bundle\") pod \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\" (UID: \"d4b0b269-0edf-4d86-be61-d2a6c6b75701\") " Jul 22 16:48:38.102871 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.102840 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-bundle" (OuterVolumeSpecName: "bundle") pod "d4b0b269-0edf-4d86-be61-d2a6c6b75701" (UID: "d4b0b269-0edf-4d86-be61-d2a6c6b75701"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:48:38.104214 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.104194 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4b0b269-0edf-4d86-be61-d2a6c6b75701-kube-api-access-dzvm9" (OuterVolumeSpecName: "kube-api-access-dzvm9") pod "d4b0b269-0edf-4d86-be61-d2a6c6b75701" (UID: "d4b0b269-0edf-4d86-be61-d2a6c6b75701"). InnerVolumeSpecName "kube-api-access-dzvm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:48:38.107886 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.107854 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-util" (OuterVolumeSpecName: "util") pod "d4b0b269-0edf-4d86-be61-d2a6c6b75701" (UID: "d4b0b269-0edf-4d86-be61-d2a6c6b75701"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:48:38.203335 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.203285 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzvm9\" (UniqueName: \"kubernetes.io/projected/d4b0b269-0edf-4d86-be61-d2a6c6b75701-kube-api-access-dzvm9\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:38.203335 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.203307 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:38.203335 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.203317 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4b0b269-0edf-4d86-be61-d2a6c6b75701-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:38.872519 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.872479 2581 generic.go:356] "Generic (PLEG): container finished" podID="1cc39293-34d0-408d-910f-79c8f4a3295d" containerID="fad221f656cbe969ba223e29f810748c2dcce6568311d6cc582a0cb299e48846" exitCode=0 Jul 22 16:48:38.872904 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.872565 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" event={"ID":"1cc39293-34d0-408d-910f-79c8f4a3295d","Type":"ContainerDied","Data":"fad221f656cbe969ba223e29f810748c2dcce6568311d6cc582a0cb299e48846"} Jul 22 16:48:38.874350 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.874335 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" Jul 22 16:48:38.874412 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.874388 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td" event={"ID":"d4b0b269-0edf-4d86-be61-d2a6c6b75701","Type":"ContainerDied","Data":"86ec436a24a1ce5d12f16b13a494744502a3d3daa30daa82d9a0f4bf659349d8"} Jul 22 16:48:38.874448 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:38.874426 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86ec436a24a1ce5d12f16b13a494744502a3d3daa30daa82d9a0f4bf659349d8" Jul 22 16:48:39.033641 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.033617 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:39.036172 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.036152 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:39.108769 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.108743 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-util\") pod \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " Jul 22 16:48:39.108907 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.108842 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-bundle\") pod \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " Jul 22 16:48:39.108907 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.108879 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvdcw\" (UniqueName: \"kubernetes.io/projected/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-kube-api-access-vvdcw\") pod \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\" (UID: \"8dacb29f-cf97-4dc0-bd83-7a619c3f6085\") " Jul 22 16:48:39.109389 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.109340 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-bundle" (OuterVolumeSpecName: "bundle") pod "8dacb29f-cf97-4dc0-bd83-7a619c3f6085" (UID: "8dacb29f-cf97-4dc0-bd83-7a619c3f6085"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:48:39.110982 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.110955 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-kube-api-access-vvdcw" (OuterVolumeSpecName: "kube-api-access-vvdcw") pod "8dacb29f-cf97-4dc0-bd83-7a619c3f6085" (UID: "8dacb29f-cf97-4dc0-bd83-7a619c3f6085"). InnerVolumeSpecName "kube-api-access-vvdcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:48:39.113828 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.113807 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-util" (OuterVolumeSpecName: "util") pod "8dacb29f-cf97-4dc0-bd83-7a619c3f6085" (UID: "8dacb29f-cf97-4dc0-bd83-7a619c3f6085"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:48:39.210196 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.210138 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-bundle\") pod \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " Jul 22 16:48:39.210276 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.210217 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-util\") pod \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " Jul 22 16:48:39.210276 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.210256 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28d9l\" (UniqueName: \"kubernetes.io/projected/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-kube-api-access-28d9l\") pod \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\" (UID: \"2f0ba89f-d2fe-4e82-af1c-58d794054c9c\") " Jul 22 16:48:39.210567 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.210542 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:39.210567 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.210568 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvdcw\" (UniqueName: \"kubernetes.io/projected/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-kube-api-access-vvdcw\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:39.210730 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.210582 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8dacb29f-cf97-4dc0-bd83-7a619c3f6085-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:39.210730 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.210632 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-bundle" (OuterVolumeSpecName: "bundle") pod "2f0ba89f-d2fe-4e82-af1c-58d794054c9c" (UID: "2f0ba89f-d2fe-4e82-af1c-58d794054c9c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:48:39.212162 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.212139 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-kube-api-access-28d9l" (OuterVolumeSpecName: "kube-api-access-28d9l") pod "2f0ba89f-d2fe-4e82-af1c-58d794054c9c" (UID: "2f0ba89f-d2fe-4e82-af1c-58d794054c9c"). InnerVolumeSpecName "kube-api-access-28d9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:48:39.215455 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.215431 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-util" (OuterVolumeSpecName: "util") pod "2f0ba89f-d2fe-4e82-af1c-58d794054c9c" (UID: "2f0ba89f-d2fe-4e82-af1c-58d794054c9c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:48:39.311007 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.310977 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:39.311007 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.311007 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28d9l\" (UniqueName: \"kubernetes.io/projected/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-kube-api-access-28d9l\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:39.311007 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.311017 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f0ba89f-d2fe-4e82-af1c-58d794054c9c-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:39.881048 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.880988 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" Jul 22 16:48:39.881048 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.881007 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj" event={"ID":"2f0ba89f-d2fe-4e82-af1c-58d794054c9c","Type":"ContainerDied","Data":"cf66b8fe9d4eab130f5ca94a30e57652986d49250600056d14b982e8665bcc78"} Jul 22 16:48:39.881530 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.881076 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf66b8fe9d4eab130f5ca94a30e57652986d49250600056d14b982e8665bcc78" Jul 22 16:48:39.883242 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.883221 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" Jul 22 16:48:39.883350 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.883277 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf" event={"ID":"8dacb29f-cf97-4dc0-bd83-7a619c3f6085","Type":"ContainerDied","Data":"ac4b99daa72444efb57ae33201fefb15bd0a36589f96ad6b4c449d78c9f99b97"} Jul 22 16:48:39.883350 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:39.883311 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac4b99daa72444efb57ae33201fefb15bd0a36589f96ad6b4c449d78c9f99b97" Jul 22 16:48:40.008748 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.008725 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:40.117270 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.117241 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-bundle\") pod \"1cc39293-34d0-408d-910f-79c8f4a3295d\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " Jul 22 16:48:40.117437 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.117303 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-util\") pod \"1cc39293-34d0-408d-910f-79c8f4a3295d\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " Jul 22 16:48:40.117437 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.117334 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vwgl\" (UniqueName: \"kubernetes.io/projected/1cc39293-34d0-408d-910f-79c8f4a3295d-kube-api-access-9vwgl\") pod \"1cc39293-34d0-408d-910f-79c8f4a3295d\" (UID: \"1cc39293-34d0-408d-910f-79c8f4a3295d\") " Jul 22 16:48:40.117816 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.117793 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-bundle" (OuterVolumeSpecName: "bundle") pod "1cc39293-34d0-408d-910f-79c8f4a3295d" (UID: "1cc39293-34d0-408d-910f-79c8f4a3295d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:48:40.119337 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.119305 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc39293-34d0-408d-910f-79c8f4a3295d-kube-api-access-9vwgl" (OuterVolumeSpecName: "kube-api-access-9vwgl") pod "1cc39293-34d0-408d-910f-79c8f4a3295d" (UID: "1cc39293-34d0-408d-910f-79c8f4a3295d"). InnerVolumeSpecName "kube-api-access-9vwgl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:48:40.122572 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.122550 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-util" (OuterVolumeSpecName: "util") pod "1cc39293-34d0-408d-910f-79c8f4a3295d" (UID: "1cc39293-34d0-408d-910f-79c8f4a3295d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:48:40.218806 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.218750 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:40.218806 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.218772 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vwgl\" (UniqueName: \"kubernetes.io/projected/1cc39293-34d0-408d-910f-79c8f4a3295d-kube-api-access-9vwgl\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:40.218806 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.218782 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cc39293-34d0-408d-910f-79c8f4a3295d-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:40.890833 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.890807 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" Jul 22 16:48:40.890833 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.890829 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s" event={"ID":"1cc39293-34d0-408d-910f-79c8f4a3295d","Type":"ContainerDied","Data":"0d769833080e88c46933f27cd5dc246be67ad74d4f31462c6f9c46c1bac59efb"} Jul 22 16:48:40.891223 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:40.890868 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d769833080e88c46933f27cd5dc246be67ad74d4f31462c6f9c46c1bac59efb" Jul 22 16:48:49.867958 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:49.867913 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8679dbbccf-84r4r" podUID="a4fbee2d-aa35-4d85-9dea-bad849069b17" containerName="console" containerID="cri-o://8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1" gracePeriod=15 Jul 22 16:48:50.116085 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.116064 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:48:50.292976 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.292953 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-config\") pod \"a4fbee2d-aa35-4d85-9dea-bad849069b17\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " Jul 22 16:48:50.293114 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.292991 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-serving-cert\") pod \"a4fbee2d-aa35-4d85-9dea-bad849069b17\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " Jul 22 16:48:50.293114 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293027 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-oauth-config\") pod \"a4fbee2d-aa35-4d85-9dea-bad849069b17\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " Jul 22 16:48:50.293114 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293058 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzcd4\" (UniqueName: \"kubernetes.io/projected/a4fbee2d-aa35-4d85-9dea-bad849069b17-kube-api-access-kzcd4\") pod \"a4fbee2d-aa35-4d85-9dea-bad849069b17\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " Jul 22 16:48:50.293114 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293110 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-trusted-ca-bundle\") pod \"a4fbee2d-aa35-4d85-9dea-bad849069b17\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " Jul 22 16:48:50.293284 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293249 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-oauth-serving-cert\") pod \"a4fbee2d-aa35-4d85-9dea-bad849069b17\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " Jul 22 16:48:50.293339 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293314 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-service-ca\") pod \"a4fbee2d-aa35-4d85-9dea-bad849069b17\" (UID: \"a4fbee2d-aa35-4d85-9dea-bad849069b17\") " Jul 22 16:48:50.293465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293437 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-config" (OuterVolumeSpecName: "console-config") pod "a4fbee2d-aa35-4d85-9dea-bad849069b17" (UID: "a4fbee2d-aa35-4d85-9dea-bad849069b17"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:48:50.293638 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293588 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a4fbee2d-aa35-4d85-9dea-bad849069b17" (UID: "a4fbee2d-aa35-4d85-9dea-bad849069b17"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:48:50.293740 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293663 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a4fbee2d-aa35-4d85-9dea-bad849069b17" (UID: "a4fbee2d-aa35-4d85-9dea-bad849069b17"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:48:50.293740 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293679 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:50.293740 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293721 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-trusted-ca-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:50.293869 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.293744 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-service-ca" (OuterVolumeSpecName: "service-ca") pod "a4fbee2d-aa35-4d85-9dea-bad849069b17" (UID: "a4fbee2d-aa35-4d85-9dea-bad849069b17"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:48:50.295107 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.295078 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a4fbee2d-aa35-4d85-9dea-bad849069b17" (UID: "a4fbee2d-aa35-4d85-9dea-bad849069b17"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:48:50.295642 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.295616 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a4fbee2d-aa35-4d85-9dea-bad849069b17" (UID: "a4fbee2d-aa35-4d85-9dea-bad849069b17"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:48:50.295642 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.295630 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fbee2d-aa35-4d85-9dea-bad849069b17-kube-api-access-kzcd4" (OuterVolumeSpecName: "kube-api-access-kzcd4") pod "a4fbee2d-aa35-4d85-9dea-bad849069b17" (UID: "a4fbee2d-aa35-4d85-9dea-bad849069b17"). InnerVolumeSpecName "kube-api-access-kzcd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:48:50.394796 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.394770 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-oauth-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:50.394796 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.394794 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a4fbee2d-aa35-4d85-9dea-bad849069b17-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:50.394907 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.394803 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:50.394907 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.394812 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a4fbee2d-aa35-4d85-9dea-bad849069b17-console-oauth-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:50.394907 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.394820 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzcd4\" (UniqueName: \"kubernetes.io/projected/a4fbee2d-aa35-4d85-9dea-bad849069b17-kube-api-access-kzcd4\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:48:50.943211 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.943176 2581 generic.go:356] "Generic (PLEG): container finished" podID="a4fbee2d-aa35-4d85-9dea-bad849069b17" containerID="8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1" exitCode=0 Jul 22 16:48:50.943593 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.943228 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8679dbbccf-84r4r" Jul 22 16:48:50.943593 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.943258 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8679dbbccf-84r4r" event={"ID":"a4fbee2d-aa35-4d85-9dea-bad849069b17","Type":"ContainerDied","Data":"8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1"} Jul 22 16:48:50.943593 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.943309 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8679dbbccf-84r4r" event={"ID":"a4fbee2d-aa35-4d85-9dea-bad849069b17","Type":"ContainerDied","Data":"709958fff21eb1aefd6660f29629e7fc0048f257a517957754ad02be68b0ca4c"} Jul 22 16:48:50.943593 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.943331 2581 scope.go:117] "RemoveContainer" containerID="8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1" Jul 22 16:48:50.952904 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.952886 2581 scope.go:117] "RemoveContainer" containerID="8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1" Jul 22 16:48:50.953145 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:48:50.953127 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1\": container with ID starting with 8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1 not found: ID does not exist" containerID="8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1" Jul 22 16:48:50.953200 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.953154 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1"} err="failed to get container status \"8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1\": rpc error: code = NotFound desc = could not find container \"8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1\": container with ID starting with 8f832dd0d778714d05b4e651b46e0e6524644c5f99e198df5e1fbbd497d7e2b1 not found: ID does not exist" Jul 22 16:48:50.969311 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.969289 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8679dbbccf-84r4r"] Jul 22 16:48:50.978475 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:50.978455 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8679dbbccf-84r4r"] Jul 22 16:48:51.618320 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:48:51.618284 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4fbee2d-aa35-4d85-9dea-bad849069b17" path="/var/lib/kubelet/pods/a4fbee2d-aa35-4d85-9dea-bad849069b17/volumes" Jul 22 16:49:08.472689 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.472609 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8"] Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473075 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4fbee2d-aa35-4d85-9dea-bad849069b17" containerName="console" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473095 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4fbee2d-aa35-4d85-9dea-bad849069b17" containerName="console" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473124 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dacb29f-cf97-4dc0-bd83-7a619c3f6085" containerName="util" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473132 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dacb29f-cf97-4dc0-bd83-7a619c3f6085" containerName="util" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473142 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b0b269-0edf-4d86-be61-d2a6c6b75701" containerName="pull" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473150 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b0b269-0edf-4d86-be61-d2a6c6b75701" containerName="pull" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473165 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dacb29f-cf97-4dc0-bd83-7a619c3f6085" containerName="pull" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473172 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dacb29f-cf97-4dc0-bd83-7a619c3f6085" containerName="pull" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473182 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0ba89f-d2fe-4e82-af1c-58d794054c9c" containerName="pull" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473189 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0ba89f-d2fe-4e82-af1c-58d794054c9c" containerName="pull" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473198 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc39293-34d0-408d-910f-79c8f4a3295d" containerName="extract" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473205 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc39293-34d0-408d-910f-79c8f4a3295d" containerName="extract" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473215 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b0b269-0edf-4d86-be61-d2a6c6b75701" containerName="extract" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473223 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b0b269-0edf-4d86-be61-d2a6c6b75701" containerName="extract" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473232 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dacb29f-cf97-4dc0-bd83-7a619c3f6085" containerName="extract" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473239 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dacb29f-cf97-4dc0-bd83-7a619c3f6085" containerName="extract" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473250 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc39293-34d0-408d-910f-79c8f4a3295d" containerName="util" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473257 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc39293-34d0-408d-910f-79c8f4a3295d" containerName="util" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473269 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b0b269-0edf-4d86-be61-d2a6c6b75701" containerName="util" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473277 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b0b269-0edf-4d86-be61-d2a6c6b75701" containerName="util" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473285 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0ba89f-d2fe-4e82-af1c-58d794054c9c" containerName="util" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473292 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0ba89f-d2fe-4e82-af1c-58d794054c9c" containerName="util" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473301 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f0ba89f-d2fe-4e82-af1c-58d794054c9c" containerName="extract" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473312 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f0ba89f-d2fe-4e82-af1c-58d794054c9c" containerName="extract" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473323 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc39293-34d0-408d-910f-79c8f4a3295d" containerName="pull" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473330 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc39293-34d0-408d-910f-79c8f4a3295d" containerName="pull" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473448 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f0ba89f-d2fe-4e82-af1c-58d794054c9c" containerName="extract" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473462 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc39293-34d0-408d-910f-79c8f4a3295d" containerName="extract" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473473 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4fbee2d-aa35-4d85-9dea-bad849069b17" containerName="console" Jul 22 16:49:08.473465 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473483 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dacb29f-cf97-4dc0-bd83-7a619c3f6085" containerName="extract" Jul 22 16:49:08.474781 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.473491 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4b0b269-0edf-4d86-be61-d2a6c6b75701" containerName="extract" Jul 22 16:49:08.476823 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.476804 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.479150 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.479121 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 22 16:49:08.479275 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.479169 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 22 16:49:08.479275 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.479222 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 16:49:08.479736 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.479721 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bfjkn\"" Jul 22 16:49:08.479822 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.479724 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 16:49:08.483204 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.483183 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8"] Jul 22 16:49:08.544041 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.544017 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6843d17c-b31d-4c13-b84b-bd97fe892592-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wrrx8\" (UID: \"6843d17c-b31d-4c13-b84b-bd97fe892592\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.544154 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.544050 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrkdx\" (UniqueName: \"kubernetes.io/projected/6843d17c-b31d-4c13-b84b-bd97fe892592-kube-api-access-lrkdx\") pod \"kuadrant-console-plugin-66b5774985-wrrx8\" (UID: \"6843d17c-b31d-4c13-b84b-bd97fe892592\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.544154 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.544078 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6843d17c-b31d-4c13-b84b-bd97fe892592-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wrrx8\" (UID: \"6843d17c-b31d-4c13-b84b-bd97fe892592\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.644635 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.644609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6843d17c-b31d-4c13-b84b-bd97fe892592-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wrrx8\" (UID: \"6843d17c-b31d-4c13-b84b-bd97fe892592\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.644789 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.644644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrkdx\" (UniqueName: \"kubernetes.io/projected/6843d17c-b31d-4c13-b84b-bd97fe892592-kube-api-access-lrkdx\") pod \"kuadrant-console-plugin-66b5774985-wrrx8\" (UID: \"6843d17c-b31d-4c13-b84b-bd97fe892592\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.644789 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.644694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6843d17c-b31d-4c13-b84b-bd97fe892592-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wrrx8\" (UID: \"6843d17c-b31d-4c13-b84b-bd97fe892592\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.645310 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.645287 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6843d17c-b31d-4c13-b84b-bd97fe892592-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wrrx8\" (UID: \"6843d17c-b31d-4c13-b84b-bd97fe892592\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.647225 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.647204 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6843d17c-b31d-4c13-b84b-bd97fe892592-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wrrx8\" (UID: \"6843d17c-b31d-4c13-b84b-bd97fe892592\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.655889 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.655860 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrkdx\" (UniqueName: \"kubernetes.io/projected/6843d17c-b31d-4c13-b84b-bd97fe892592-kube-api-access-lrkdx\") pod \"kuadrant-console-plugin-66b5774985-wrrx8\" (UID: \"6843d17c-b31d-4c13-b84b-bd97fe892592\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.786185 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.786117 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" Jul 22 16:49:08.928396 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:08.928358 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8"] Jul 22 16:49:08.930971 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:49:08.930945 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6843d17c_b31d_4c13_b84b_bd97fe892592.slice/crio-6ee0f57017ee5fc5e3e55a2c319ae5976866dda94c6a1b51a40824cea0ab4e20 WatchSource:0}: Error finding container 6ee0f57017ee5fc5e3e55a2c319ae5976866dda94c6a1b51a40824cea0ab4e20: Status 404 returned error can't find the container with id 6ee0f57017ee5fc5e3e55a2c319ae5976866dda94c6a1b51a40824cea0ab4e20 Jul 22 16:49:09.034380 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:09.034321 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" event={"ID":"6843d17c-b31d-4c13-b84b-bd97fe892592","Type":"ContainerStarted","Data":"6ee0f57017ee5fc5e3e55a2c319ae5976866dda94c6a1b51a40824cea0ab4e20"} Jul 22 16:49:31.168720 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:31.168682 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" event={"ID":"6843d17c-b31d-4c13-b84b-bd97fe892592","Type":"ContainerStarted","Data":"8f70f7fbd5d63bca07065877e91610a95439b0c331a910183cddfe12054ca95c"} Jul 22 16:49:31.184569 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:49:31.184526 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wrrx8" podStartSLOduration=1.550264634 podStartE2EDuration="23.184512901s" totalImagesPullingTime="21.634248267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:49:31.183648633 +0000 UTC m=+558.094305803" watchObservedRunningTime="2026-07-22 16:49:31.184512901 +0000 UTC m=+558.095170073" Jul 22 16:50:06.982119 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:06.982085 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-lsj4h"] Jul 22 16:50:07.031012 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.030984 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-lsj4h"] Jul 22 16:50:07.031163 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.031070 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lsj4h" Jul 22 16:50:07.033861 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.033838 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-w9npl\"" Jul 22 16:50:07.108431 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.108401 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-m9zn2"] Jul 22 16:50:07.111801 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.111784 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m9zn2" Jul 22 16:50:07.118569 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.118550 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m9zn2"] Jul 22 16:50:07.135087 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.135066 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llvxm\" (UniqueName: \"kubernetes.io/projected/5d015c7c-b79d-4523-865d-d72d77a971f0-kube-api-access-llvxm\") pod \"authorino-58cd955675-lsj4h\" (UID: \"5d015c7c-b79d-4523-865d-d72d77a971f0\") " pod="kuadrant-system/authorino-58cd955675-lsj4h" Jul 22 16:50:07.235870 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.235818 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llvxm\" (UniqueName: \"kubernetes.io/projected/5d015c7c-b79d-4523-865d-d72d77a971f0-kube-api-access-llvxm\") pod \"authorino-58cd955675-lsj4h\" (UID: \"5d015c7c-b79d-4523-865d-d72d77a971f0\") " pod="kuadrant-system/authorino-58cd955675-lsj4h" Jul 22 16:50:07.235870 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.235849 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jlrk\" (UniqueName: \"kubernetes.io/projected/a3127a16-35aa-4691-8322-fb8ee404a35e-kube-api-access-7jlrk\") pod \"authorino-75d556bd77-m9zn2\" (UID: \"a3127a16-35aa-4691-8322-fb8ee404a35e\") " pod="kuadrant-system/authorino-75d556bd77-m9zn2" Jul 22 16:50:07.244153 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.244128 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llvxm\" (UniqueName: \"kubernetes.io/projected/5d015c7c-b79d-4523-865d-d72d77a971f0-kube-api-access-llvxm\") pod \"authorino-58cd955675-lsj4h\" (UID: \"5d015c7c-b79d-4523-865d-d72d77a971f0\") " pod="kuadrant-system/authorino-58cd955675-lsj4h" Jul 22 16:50:07.336180 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.336158 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jlrk\" (UniqueName: \"kubernetes.io/projected/a3127a16-35aa-4691-8322-fb8ee404a35e-kube-api-access-7jlrk\") pod \"authorino-75d556bd77-m9zn2\" (UID: \"a3127a16-35aa-4691-8322-fb8ee404a35e\") " pod="kuadrant-system/authorino-75d556bd77-m9zn2" Jul 22 16:50:07.339308 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.339280 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lsj4h" Jul 22 16:50:07.347116 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.347094 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jlrk\" (UniqueName: \"kubernetes.io/projected/a3127a16-35aa-4691-8322-fb8ee404a35e-kube-api-access-7jlrk\") pod \"authorino-75d556bd77-m9zn2\" (UID: \"a3127a16-35aa-4691-8322-fb8ee404a35e\") " pod="kuadrant-system/authorino-75d556bd77-m9zn2" Jul 22 16:50:07.420819 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.420555 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m9zn2" Jul 22 16:50:07.472120 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.471923 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-lsj4h"] Jul 22 16:50:07.475034 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:50:07.474977 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d015c7c_b79d_4523_865d_d72d77a971f0.slice/crio-edec88c3dd485dbad047b57467e993c2908f789c87243d5b2fc3bc757aa21994 WatchSource:0}: Error finding container edec88c3dd485dbad047b57467e993c2908f789c87243d5b2fc3bc757aa21994: Status 404 returned error can't find the container with id edec88c3dd485dbad047b57467e993c2908f789c87243d5b2fc3bc757aa21994 Jul 22 16:50:07.552000 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:07.551976 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m9zn2"] Jul 22 16:50:07.553925 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:50:07.553895 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3127a16_35aa_4691_8322_fb8ee404a35e.slice/crio-0aed30499779a11800fe2d17dc89488195db140534f1d84996047c832d0601d4 WatchSource:0}: Error finding container 0aed30499779a11800fe2d17dc89488195db140534f1d84996047c832d0601d4: Status 404 returned error can't find the container with id 0aed30499779a11800fe2d17dc89488195db140534f1d84996047c832d0601d4 Jul 22 16:50:08.355778 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:08.355708 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m9zn2" event={"ID":"a3127a16-35aa-4691-8322-fb8ee404a35e","Type":"ContainerStarted","Data":"0aed30499779a11800fe2d17dc89488195db140534f1d84996047c832d0601d4"} Jul 22 16:50:08.359492 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:08.359424 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lsj4h" event={"ID":"5d015c7c-b79d-4523-865d-d72d77a971f0","Type":"ContainerStarted","Data":"edec88c3dd485dbad047b57467e993c2908f789c87243d5b2fc3bc757aa21994"} Jul 22 16:50:11.380895 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:11.380809 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lsj4h" event={"ID":"5d015c7c-b79d-4523-865d-d72d77a971f0","Type":"ContainerStarted","Data":"58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085"} Jul 22 16:50:11.382263 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:11.382240 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m9zn2" event={"ID":"a3127a16-35aa-4691-8322-fb8ee404a35e","Type":"ContainerStarted","Data":"1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76"} Jul 22 16:50:11.395164 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:11.395081 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-lsj4h" podStartSLOduration=1.760652192 podStartE2EDuration="5.395065082s" totalImagesPullingTime="3.63441289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:50:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:50:11.395009352 +0000 UTC m=+598.305666523" watchObservedRunningTime="2026-07-22 16:50:11.395065082 +0000 UTC m=+598.305722251" Jul 22 16:50:11.410914 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:11.410865 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-m9zn2" podStartSLOduration=0.844075403 podStartE2EDuration="4.410848154s" totalImagesPullingTime="3.566772751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:50:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:50:11.4090533 +0000 UTC m=+598.319710473" watchObservedRunningTime="2026-07-22 16:50:11.410848154 +0000 UTC m=+598.321505327" Jul 22 16:50:11.431453 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:11.431425 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-lsj4h"] Jul 22 16:50:13.393105 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:13.393056 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-lsj4h" podUID="5d015c7c-b79d-4523-865d-d72d77a971f0" containerName="authorino" containerID="cri-o://58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085" gracePeriod=30 Jul 22 16:50:13.562207 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:13.562178 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 16:50:13.568966 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:13.568944 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 16:50:13.651615 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:13.651552 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lsj4h" Jul 22 16:50:13.794294 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:13.794260 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llvxm\" (UniqueName: \"kubernetes.io/projected/5d015c7c-b79d-4523-865d-d72d77a971f0-kube-api-access-llvxm\") pod \"5d015c7c-b79d-4523-865d-d72d77a971f0\" (UID: \"5d015c7c-b79d-4523-865d-d72d77a971f0\") " Jul 22 16:50:13.796275 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:13.796257 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d015c7c-b79d-4523-865d-d72d77a971f0-kube-api-access-llvxm" (OuterVolumeSpecName: "kube-api-access-llvxm") pod "5d015c7c-b79d-4523-865d-d72d77a971f0" (UID: "5d015c7c-b79d-4523-865d-d72d77a971f0"). InnerVolumeSpecName "kube-api-access-llvxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:50:13.895145 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:13.895120 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llvxm\" (UniqueName: \"kubernetes.io/projected/5d015c7c-b79d-4523-865d-d72d77a971f0-kube-api-access-llvxm\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:50:14.399398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:14.399348 2581 generic.go:356] "Generic (PLEG): container finished" podID="5d015c7c-b79d-4523-865d-d72d77a971f0" containerID="58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085" exitCode=0 Jul 22 16:50:14.399398 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:14.399404 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-lsj4h" Jul 22 16:50:14.399898 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:14.399436 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lsj4h" event={"ID":"5d015c7c-b79d-4523-865d-d72d77a971f0","Type":"ContainerDied","Data":"58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085"} Jul 22 16:50:14.399898 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:14.399484 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-lsj4h" event={"ID":"5d015c7c-b79d-4523-865d-d72d77a971f0","Type":"ContainerDied","Data":"edec88c3dd485dbad047b57467e993c2908f789c87243d5b2fc3bc757aa21994"} Jul 22 16:50:14.399898 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:14.399510 2581 scope.go:117] "RemoveContainer" containerID="58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085" Jul 22 16:50:14.408255 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:14.408239 2581 scope.go:117] "RemoveContainer" containerID="58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085" Jul 22 16:50:14.408535 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:50:14.408515 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085\": container with ID starting with 58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085 not found: ID does not exist" containerID="58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085" Jul 22 16:50:14.408605 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:14.408544 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085"} err="failed to get container status \"58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085\": rpc error: code = NotFound desc = could not find container \"58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085\": container with ID starting with 58defc9811c6af93a92a84b9c252d13d365cfe841a60e5eb39744633988c4085 not found: ID does not exist" Jul 22 16:50:14.420487 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:14.420462 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-lsj4h"] Jul 22 16:50:14.424054 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:14.424035 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-lsj4h"] Jul 22 16:50:15.619054 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:15.619018 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d015c7c-b79d-4523-865d-d72d77a971f0" path="/var/lib/kubelet/pods/5d015c7c-b79d-4523-865d-d72d77a971f0/volumes" Jul 22 16:50:37.015065 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.014987 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xcl7j"] Jul 22 16:50:37.015526 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.015310 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d015c7c-b79d-4523-865d-d72d77a971f0" containerName="authorino" Jul 22 16:50:37.015526 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.015322 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d015c7c-b79d-4523-865d-d72d77a971f0" containerName="authorino" Jul 22 16:50:37.015526 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.015396 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d015c7c-b79d-4523-865d-d72d77a971f0" containerName="authorino" Jul 22 16:50:37.019613 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.019595 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xcl7j" Jul 22 16:50:37.033643 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.033618 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xcl7j"] Jul 22 16:50:37.169767 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.169741 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts5rr\" (UniqueName: \"kubernetes.io/projected/a428c59c-e5a0-4766-8abf-5fb0cf02ee8d-kube-api-access-ts5rr\") pod \"authorino-5d4b67c446-xcl7j\" (UID: \"a428c59c-e5a0-4766-8abf-5fb0cf02ee8d\") " pod="kuadrant-system/authorino-5d4b67c446-xcl7j" Jul 22 16:50:37.218446 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.218417 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xcl7j"] Jul 22 16:50:37.218648 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:50:37.218619 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ts5rr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-xcl7j" podUID="a428c59c-e5a0-4766-8abf-5fb0cf02ee8d" Jul 22 16:50:37.245401 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.245374 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-584b547f58-mddrn"] Jul 22 16:50:37.248667 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.248649 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-584b547f58-mddrn" Jul 22 16:50:37.255602 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.255580 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-584b547f58-mddrn"] Jul 22 16:50:37.271078 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.271028 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts5rr\" (UniqueName: \"kubernetes.io/projected/a428c59c-e5a0-4766-8abf-5fb0cf02ee8d-kube-api-access-ts5rr\") pod \"authorino-5d4b67c446-xcl7j\" (UID: \"a428c59c-e5a0-4766-8abf-5fb0cf02ee8d\") " pod="kuadrant-system/authorino-5d4b67c446-xcl7j" Jul 22 16:50:37.279032 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.279015 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts5rr\" (UniqueName: \"kubernetes.io/projected/a428c59c-e5a0-4766-8abf-5fb0cf02ee8d-kube-api-access-ts5rr\") pod \"authorino-5d4b67c446-xcl7j\" (UID: \"a428c59c-e5a0-4766-8abf-5fb0cf02ee8d\") " pod="kuadrant-system/authorino-5d4b67c446-xcl7j" Jul 22 16:50:37.371596 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.371571 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hkpg\" (UniqueName: \"kubernetes.io/projected/0620111f-317c-4b6e-864c-f82faf89b1ce-kube-api-access-6hkpg\") pod \"authorino-584b547f58-mddrn\" (UID: \"0620111f-317c-4b6e-864c-f82faf89b1ce\") " pod="kuadrant-system/authorino-584b547f58-mddrn" Jul 22 16:50:37.441284 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.441259 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-584b547f58-mddrn"] Jul 22 16:50:37.441479 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:50:37.441456 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-6hkpg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-584b547f58-mddrn" podUID="0620111f-317c-4b6e-864c-f82faf89b1ce" Jul 22 16:50:37.472461 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.472435 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7fdf59ddf8-9ltqb"] Jul 22 16:50:37.472673 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.472653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hkpg\" (UniqueName: \"kubernetes.io/projected/0620111f-317c-4b6e-864c-f82faf89b1ce-kube-api-access-6hkpg\") pod \"authorino-584b547f58-mddrn\" (UID: \"0620111f-317c-4b6e-864c-f82faf89b1ce\") " pod="kuadrant-system/authorino-584b547f58-mddrn" Jul 22 16:50:37.475786 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.475771 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" Jul 22 16:50:37.478194 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.478178 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 22 16:50:37.484235 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.484217 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fdf59ddf8-9ltqb"] Jul 22 16:50:37.485673 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.485653 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hkpg\" (UniqueName: \"kubernetes.io/projected/0620111f-317c-4b6e-864c-f82faf89b1ce-kube-api-access-6hkpg\") pod \"authorino-584b547f58-mddrn\" (UID: \"0620111f-317c-4b6e-864c-f82faf89b1ce\") " pod="kuadrant-system/authorino-584b547f58-mddrn" Jul 22 16:50:37.516569 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.516549 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-584b547f58-mddrn" Jul 22 16:50:37.516649 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.516557 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xcl7j" Jul 22 16:50:37.523006 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.522961 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-584b547f58-mddrn" Jul 22 16:50:37.531659 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.531314 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xcl7j" Jul 22 16:50:37.573298 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.573273 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/845c8797-4295-4a10-b121-3f0fe365d977-tls-cert\") pod \"authorino-7fdf59ddf8-9ltqb\" (UID: \"845c8797-4295-4a10-b121-3f0fe365d977\") " pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" Jul 22 16:50:37.573419 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.573333 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd2n9\" (UniqueName: \"kubernetes.io/projected/845c8797-4295-4a10-b121-3f0fe365d977-kube-api-access-jd2n9\") pod \"authorino-7fdf59ddf8-9ltqb\" (UID: \"845c8797-4295-4a10-b121-3f0fe365d977\") " pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" Jul 22 16:50:37.673907 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.673884 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hkpg\" (UniqueName: \"kubernetes.io/projected/0620111f-317c-4b6e-864c-f82faf89b1ce-kube-api-access-6hkpg\") pod \"0620111f-317c-4b6e-864c-f82faf89b1ce\" (UID: \"0620111f-317c-4b6e-864c-f82faf89b1ce\") " Jul 22 16:50:37.673996 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.673919 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts5rr\" (UniqueName: \"kubernetes.io/projected/a428c59c-e5a0-4766-8abf-5fb0cf02ee8d-kube-api-access-ts5rr\") pod \"a428c59c-e5a0-4766-8abf-5fb0cf02ee8d\" (UID: \"a428c59c-e5a0-4766-8abf-5fb0cf02ee8d\") " Jul 22 16:50:37.674044 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.674025 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd2n9\" (UniqueName: \"kubernetes.io/projected/845c8797-4295-4a10-b121-3f0fe365d977-kube-api-access-jd2n9\") pod \"authorino-7fdf59ddf8-9ltqb\" (UID: \"845c8797-4295-4a10-b121-3f0fe365d977\") " pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" Jul 22 16:50:37.674081 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.674059 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/845c8797-4295-4a10-b121-3f0fe365d977-tls-cert\") pod \"authorino-7fdf59ddf8-9ltqb\" (UID: \"845c8797-4295-4a10-b121-3f0fe365d977\") " pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" Jul 22 16:50:37.676048 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.676022 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0620111f-317c-4b6e-864c-f82faf89b1ce-kube-api-access-6hkpg" (OuterVolumeSpecName: "kube-api-access-6hkpg") pod "0620111f-317c-4b6e-864c-f82faf89b1ce" (UID: "0620111f-317c-4b6e-864c-f82faf89b1ce"). InnerVolumeSpecName "kube-api-access-6hkpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:50:37.676329 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.676128 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a428c59c-e5a0-4766-8abf-5fb0cf02ee8d-kube-api-access-ts5rr" (OuterVolumeSpecName: "kube-api-access-ts5rr") pod "a428c59c-e5a0-4766-8abf-5fb0cf02ee8d" (UID: "a428c59c-e5a0-4766-8abf-5fb0cf02ee8d"). InnerVolumeSpecName "kube-api-access-ts5rr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:50:37.676329 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.676264 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/845c8797-4295-4a10-b121-3f0fe365d977-tls-cert\") pod \"authorino-7fdf59ddf8-9ltqb\" (UID: \"845c8797-4295-4a10-b121-3f0fe365d977\") " pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" Jul 22 16:50:37.681796 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.681776 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd2n9\" (UniqueName: \"kubernetes.io/projected/845c8797-4295-4a10-b121-3f0fe365d977-kube-api-access-jd2n9\") pod \"authorino-7fdf59ddf8-9ltqb\" (UID: \"845c8797-4295-4a10-b121-3f0fe365d977\") " pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" Jul 22 16:50:37.775393 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.775326 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hkpg\" (UniqueName: \"kubernetes.io/projected/0620111f-317c-4b6e-864c-f82faf89b1ce-kube-api-access-6hkpg\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:50:37.775393 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.775348 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts5rr\" (UniqueName: \"kubernetes.io/projected/a428c59c-e5a0-4766-8abf-5fb0cf02ee8d-kube-api-access-ts5rr\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:50:37.785323 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.785305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" Jul 22 16:50:37.916475 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:37.916443 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fdf59ddf8-9ltqb"] Jul 22 16:50:37.919480 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:50:37.919450 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod845c8797_4295_4a10_b121_3f0fe365d977.slice/crio-94d98ddeced8ff5ec9c81e9313023171fe3226f019b022a779f37f6e707756e7 WatchSource:0}: Error finding container 94d98ddeced8ff5ec9c81e9313023171fe3226f019b022a779f37f6e707756e7: Status 404 returned error can't find the container with id 94d98ddeced8ff5ec9c81e9313023171fe3226f019b022a779f37f6e707756e7 Jul 22 16:50:38.523638 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.523571 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xcl7j" Jul 22 16:50:38.523638 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.523602 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-584b547f58-mddrn" Jul 22 16:50:38.524013 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.523562 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" event={"ID":"845c8797-4295-4a10-b121-3f0fe365d977","Type":"ContainerStarted","Data":"d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1"} Jul 22 16:50:38.524013 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.523693 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" event={"ID":"845c8797-4295-4a10-b121-3f0fe365d977","Type":"ContainerStarted","Data":"94d98ddeced8ff5ec9c81e9313023171fe3226f019b022a779f37f6e707756e7"} Jul 22 16:50:38.548400 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.548350 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" podStartSLOduration=1.232897714 podStartE2EDuration="1.548336461s" totalImagesPullingTime="315.438747ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:50:38.545936378 +0000 UTC m=+625.456593564" watchObservedRunningTime="2026-07-22 16:50:38.548336461 +0000 UTC m=+625.458993634" Jul 22 16:50:38.572921 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.572897 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m9zn2"] Jul 22 16:50:38.573070 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.573051 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-m9zn2" podUID="a3127a16-35aa-4691-8322-fb8ee404a35e" containerName="authorino" containerID="cri-o://1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76" gracePeriod=30 Jul 22 16:50:38.582442 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.582395 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xcl7j"] Jul 22 16:50:38.585699 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.585672 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xcl7j"] Jul 22 16:50:38.608280 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.608253 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-584b547f58-mddrn"] Jul 22 16:50:38.610121 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.610101 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-584b547f58-mddrn"] Jul 22 16:50:38.810698 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.810677 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m9zn2" Jul 22 16:50:38.984131 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.984104 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jlrk\" (UniqueName: \"kubernetes.io/projected/a3127a16-35aa-4691-8322-fb8ee404a35e-kube-api-access-7jlrk\") pod \"a3127a16-35aa-4691-8322-fb8ee404a35e\" (UID: \"a3127a16-35aa-4691-8322-fb8ee404a35e\") " Jul 22 16:50:38.986285 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:38.986252 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3127a16-35aa-4691-8322-fb8ee404a35e-kube-api-access-7jlrk" (OuterVolumeSpecName: "kube-api-access-7jlrk") pod "a3127a16-35aa-4691-8322-fb8ee404a35e" (UID: "a3127a16-35aa-4691-8322-fb8ee404a35e"). InnerVolumeSpecName "kube-api-access-7jlrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:50:39.084908 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.084848 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jlrk\" (UniqueName: \"kubernetes.io/projected/a3127a16-35aa-4691-8322-fb8ee404a35e-kube-api-access-7jlrk\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:50:39.530702 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.530673 2581 generic.go:356] "Generic (PLEG): container finished" podID="a3127a16-35aa-4691-8322-fb8ee404a35e" containerID="1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76" exitCode=0 Jul 22 16:50:39.531083 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.530715 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m9zn2" Jul 22 16:50:39.531083 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.530757 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m9zn2" event={"ID":"a3127a16-35aa-4691-8322-fb8ee404a35e","Type":"ContainerDied","Data":"1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76"} Jul 22 16:50:39.531083 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.530804 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m9zn2" event={"ID":"a3127a16-35aa-4691-8322-fb8ee404a35e","Type":"ContainerDied","Data":"0aed30499779a11800fe2d17dc89488195db140534f1d84996047c832d0601d4"} Jul 22 16:50:39.531083 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.530835 2581 scope.go:117] "RemoveContainer" containerID="1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76" Jul 22 16:50:39.543105 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.543090 2581 scope.go:117] "RemoveContainer" containerID="1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76" Jul 22 16:50:39.543344 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:50:39.543325 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76\": container with ID starting with 1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76 not found: ID does not exist" containerID="1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76" Jul 22 16:50:39.543439 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.543354 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76"} err="failed to get container status \"1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76\": rpc error: code = NotFound desc = could not find container \"1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76\": container with ID starting with 1bbc09cccd54eb13755a4773457cfda51e8e4a8c86af666f524934e6b6a18b76 not found: ID does not exist" Jul 22 16:50:39.555774 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.555730 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m9zn2"] Jul 22 16:50:39.558153 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.558129 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m9zn2"] Jul 22 16:50:39.618647 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.618618 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0620111f-317c-4b6e-864c-f82faf89b1ce" path="/var/lib/kubelet/pods/0620111f-317c-4b6e-864c-f82faf89b1ce/volumes" Jul 22 16:50:39.618852 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.618839 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3127a16-35aa-4691-8322-fb8ee404a35e" path="/var/lib/kubelet/pods/a3127a16-35aa-4691-8322-fb8ee404a35e/volumes" Jul 22 16:50:39.619122 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:50:39.619111 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a428c59c-e5a0-4766-8abf-5fb0cf02ee8d" path="/var/lib/kubelet/pods/a428c59c-e5a0-4766-8abf-5fb0cf02ee8d/volumes" Jul 22 16:52:12.198569 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.198480 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m6988"] Jul 22 16:52:12.199083 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.199016 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3127a16-35aa-4691-8322-fb8ee404a35e" containerName="authorino" Jul 22 16:52:12.199083 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.199036 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3127a16-35aa-4691-8322-fb8ee404a35e" containerName="authorino" Jul 22 16:52:12.199206 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.199155 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3127a16-35aa-4691-8322-fb8ee404a35e" containerName="authorino" Jul 22 16:52:12.202441 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.202417 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.205456 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.205437 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7d4dj\"" Jul 22 16:52:12.205589 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.205444 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 22 16:52:12.205589 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.205478 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 16:52:12.205589 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.205510 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 16:52:12.212034 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.212012 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m6988"] Jul 22 16:52:12.302768 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.302746 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.302910 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.302789 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rscsn\" (UniqueName: \"kubernetes.io/projected/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-kube-api-access-rscsn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.302910 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.302818 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.302910 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.302874 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.302910 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.302908 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.303090 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.302948 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.403525 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.403489 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.403670 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.403556 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rscsn\" (UniqueName: \"kubernetes.io/projected/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-kube-api-access-rscsn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.403670 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.403585 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.403670 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.403617 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.403670 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.403650 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.403878 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.403701 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.404003 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.403983 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.404077 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.404033 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.404136 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.404093 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.405809 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.405790 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.406060 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.406044 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.411303 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.411285 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rscsn\" (UniqueName: \"kubernetes.io/projected/10ff4d4b-de14-47f4-bb4c-cebc4da6f860-kube-api-access-rscsn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m6988\" (UID: \"10ff4d4b-de14-47f4-bb4c-cebc4da6f860\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.513519 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.513497 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:12.641841 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.641819 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m6988"] Jul 22 16:52:12.643931 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:52:12.643902 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ff4d4b_de14_47f4_bb4c_cebc4da6f860.slice/crio-b96cc233990e05c1e0b00dd0a69a8bb88abee82302e8bbe759a89e135ffdfb6a WatchSource:0}: Error finding container b96cc233990e05c1e0b00dd0a69a8bb88abee82302e8bbe759a89e135ffdfb6a: Status 404 returned error can't find the container with id b96cc233990e05c1e0b00dd0a69a8bb88abee82302e8bbe759a89e135ffdfb6a Jul 22 16:52:12.645404 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:12.645386 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:52:13.003182 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:13.003141 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" event={"ID":"10ff4d4b-de14-47f4-bb4c-cebc4da6f860","Type":"ContainerStarted","Data":"b96cc233990e05c1e0b00dd0a69a8bb88abee82302e8bbe759a89e135ffdfb6a"} Jul 22 16:52:14.010354 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:14.010324 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" event={"ID":"10ff4d4b-de14-47f4-bb4c-cebc4da6f860","Type":"ContainerStarted","Data":"7cae08ec2e924faf688460fd4faeffb604aa16849cc416ef7fe9231e81af4ed8"} Jul 22 16:52:14.029518 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:14.029443 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" podStartSLOduration=0.907926448 podStartE2EDuration="2.029429114s" totalImagesPullingTime="1.121502666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:52:14.027891285 +0000 UTC m=+720.938548457" watchObservedRunningTime="2026-07-22 16:52:14.029429114 +0000 UTC m=+720.940086286" Jul 22 16:52:22.514444 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:22.514408 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:22.514841 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:22.514478 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:22.526719 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:22.526690 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:23.067131 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:23.067104 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m6988" Jul 22 16:52:24.291904 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.291864 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp"] Jul 22 16:52:24.295310 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.295291 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.297599 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.297581 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 22 16:52:24.313670 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.313642 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp"] Jul 22 16:52:24.399787 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.399749 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqh9l\" (UniqueName: \"kubernetes.io/projected/49d96579-47cd-4937-900a-b229e3db8830-kube-api-access-xqh9l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.399953 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.399794 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.399953 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.399820 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.399953 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.399837 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.399953 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.399877 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49d96579-47cd-4937-900a-b229e3db8830-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.400092 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.399973 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.500486 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.500455 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqh9l\" (UniqueName: \"kubernetes.io/projected/49d96579-47cd-4937-900a-b229e3db8830-kube-api-access-xqh9l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.500486 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.500492 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.500725 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.500515 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.500725 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.500531 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.500725 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.500548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49d96579-47cd-4937-900a-b229e3db8830-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.500725 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.500609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.500974 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.500947 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.501081 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.500996 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.501081 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.501043 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.502797 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.502777 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49d96579-47cd-4937-900a-b229e3db8830-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.503140 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.503123 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49d96579-47cd-4937-900a-b229e3db8830-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.509405 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.509384 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqh9l\" (UniqueName: \"kubernetes.io/projected/49d96579-47cd-4937-900a-b229e3db8830-kube-api-access-xqh9l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp\" (UID: \"49d96579-47cd-4937-900a-b229e3db8830\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.605718 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.605648 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:24.740126 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:24.740101 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp"] Jul 22 16:52:24.742294 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:52:24.742268 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d96579_47cd_4937_900a_b229e3db8830.slice/crio-b69d25072e747c2b1ca50d03aea370c4d38250fc7ccecc9df7ad4127760e5c24 WatchSource:0}: Error finding container b69d25072e747c2b1ca50d03aea370c4d38250fc7ccecc9df7ad4127760e5c24: Status 404 returned error can't find the container with id b69d25072e747c2b1ca50d03aea370c4d38250fc7ccecc9df7ad4127760e5c24 Jul 22 16:52:25.066509 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:25.066475 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" event={"ID":"49d96579-47cd-4937-900a-b229e3db8830","Type":"ContainerStarted","Data":"b69d25072e747c2b1ca50d03aea370c4d38250fc7ccecc9df7ad4127760e5c24"} Jul 22 16:52:27.078679 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:27.078595 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" event={"ID":"49d96579-47cd-4937-900a-b229e3db8830","Type":"ContainerStarted","Data":"d626e281fb547c011d9d6d516f4afed1a21d0454aa99777fe623efb4ad8e7605"} Jul 22 16:52:27.117131 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:27.117084 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" podStartSLOduration=1.052378175 podStartE2EDuration="3.117067655s" totalImagesPullingTime="2.06468948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:52:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:52:27.115672373 +0000 UTC m=+734.026329546" watchObservedRunningTime="2026-07-22 16:52:27.117067655 +0000 UTC m=+734.027724827" Jul 22 16:52:34.606278 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:34.606239 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:34.606686 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:34.606296 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:34.618321 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:34.618294 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:35.137231 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:35.137201 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp" Jul 22 16:52:44.988265 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:44.988230 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6"] Jul 22 16:52:44.993192 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:44.993172 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.000838 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.000813 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 22 16:52:45.006721 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.006690 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6"] Jul 22 16:52:45.072152 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.072129 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.072256 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.072159 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.072256 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.072177 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.072332 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.072259 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9f1c8e6-d565-4310-9dbd-e9ed034332df-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.072332 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.072295 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k26sp\" (UniqueName: \"kubernetes.io/projected/e9f1c8e6-d565-4310-9dbd-e9ed034332df-kube-api-access-k26sp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.072332 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.072315 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.173342 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.173318 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9f1c8e6-d565-4310-9dbd-e9ed034332df-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.173469 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.173356 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k26sp\" (UniqueName: \"kubernetes.io/projected/e9f1c8e6-d565-4310-9dbd-e9ed034332df-kube-api-access-k26sp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.173469 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.173401 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.173469 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.173447 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.173469 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.173464 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.173688 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.173482 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.173906 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.173887 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.173983 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.173925 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.174051 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.174008 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.176280 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.176249 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9f1c8e6-d565-4310-9dbd-e9ed034332df-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.176506 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.176489 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9f1c8e6-d565-4310-9dbd-e9ed034332df-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.181978 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.181954 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k26sp\" (UniqueName: \"kubernetes.io/projected/e9f1c8e6-d565-4310-9dbd-e9ed034332df-kube-api-access-k26sp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-thzq6\" (UID: \"e9f1c8e6-d565-4310-9dbd-e9ed034332df\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.302313 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.302256 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:45.429226 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:45.429201 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6"] Jul 22 16:52:45.431277 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:52:45.431250 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9f1c8e6_d565_4310_9dbd_e9ed034332df.slice/crio-03b29d844451c5ad3b776d1c8c958c8033c2390aa0e5d579bfba0f909422765d WatchSource:0}: Error finding container 03b29d844451c5ad3b776d1c8c958c8033c2390aa0e5d579bfba0f909422765d: Status 404 returned error can't find the container with id 03b29d844451c5ad3b776d1c8c958c8033c2390aa0e5d579bfba0f909422765d Jul 22 16:52:46.185288 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:46.185254 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" event={"ID":"e9f1c8e6-d565-4310-9dbd-e9ed034332df","Type":"ContainerStarted","Data":"f1fe31528fa54cc3082371fac72f9b03fc31f27348b76646cfabe4c1881cbb26"} Jul 22 16:52:46.185699 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:46.185296 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" event={"ID":"e9f1c8e6-d565-4310-9dbd-e9ed034332df","Type":"ContainerStarted","Data":"03b29d844451c5ad3b776d1c8c958c8033c2390aa0e5d579bfba0f909422765d"} Jul 22 16:52:46.204853 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:46.204810 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" podStartSLOduration=2.016106485 podStartE2EDuration="2.20479676s" totalImagesPullingTime="188.690275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:52:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:52:46.202704392 +0000 UTC m=+753.113361578" watchObservedRunningTime="2026-07-22 16:52:46.20479676 +0000 UTC m=+753.115453987" Jul 22 16:52:55.302954 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:55.302919 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:55.303458 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:55.302985 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:55.315769 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:55.315747 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:52:56.255011 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:52:56.254979 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-thzq6" Jul 22 16:53:37.034781 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.034696 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-689b965f47-4wx72"] Jul 22 16:53:37.038240 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.038222 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-689b965f47-4wx72" Jul 22 16:53:37.064274 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.064250 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-689b965f47-4wx72"] Jul 22 16:53:37.091171 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.091141 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fc927414-66ea-4c2a-a716-4be30de05f17-tls-cert\") pod \"authorino-689b965f47-4wx72\" (UID: \"fc927414-66ea-4c2a-a716-4be30de05f17\") " pod="kuadrant-system/authorino-689b965f47-4wx72" Jul 22 16:53:37.091280 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.091223 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dxhm\" (UniqueName: \"kubernetes.io/projected/fc927414-66ea-4c2a-a716-4be30de05f17-kube-api-access-4dxhm\") pod \"authorino-689b965f47-4wx72\" (UID: \"fc927414-66ea-4c2a-a716-4be30de05f17\") " pod="kuadrant-system/authorino-689b965f47-4wx72" Jul 22 16:53:37.192122 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.192099 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fc927414-66ea-4c2a-a716-4be30de05f17-tls-cert\") pod \"authorino-689b965f47-4wx72\" (UID: \"fc927414-66ea-4c2a-a716-4be30de05f17\") " pod="kuadrant-system/authorino-689b965f47-4wx72" Jul 22 16:53:37.192226 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.192179 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dxhm\" (UniqueName: \"kubernetes.io/projected/fc927414-66ea-4c2a-a716-4be30de05f17-kube-api-access-4dxhm\") pod \"authorino-689b965f47-4wx72\" (UID: \"fc927414-66ea-4c2a-a716-4be30de05f17\") " pod="kuadrant-system/authorino-689b965f47-4wx72" Jul 22 16:53:37.194468 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.194451 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fc927414-66ea-4c2a-a716-4be30de05f17-tls-cert\") pod \"authorino-689b965f47-4wx72\" (UID: \"fc927414-66ea-4c2a-a716-4be30de05f17\") " pod="kuadrant-system/authorino-689b965f47-4wx72" Jul 22 16:53:37.201129 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.201107 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dxhm\" (UniqueName: \"kubernetes.io/projected/fc927414-66ea-4c2a-a716-4be30de05f17-kube-api-access-4dxhm\") pod \"authorino-689b965f47-4wx72\" (UID: \"fc927414-66ea-4c2a-a716-4be30de05f17\") " pod="kuadrant-system/authorino-689b965f47-4wx72" Jul 22 16:53:37.347053 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.346995 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-689b965f47-4wx72" Jul 22 16:53:37.498528 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:37.498346 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-689b965f47-4wx72"] Jul 22 16:53:37.502111 ip-10-0-132-57 kubenswrapper[2581]: W0722 16:53:37.502085 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc927414_66ea_4c2a_a716_4be30de05f17.slice/crio-2be5a2b62b3ea6cce0c0c02092b0b46ac6cb3b45fabb46dca17ad2a470336d61 WatchSource:0}: Error finding container 2be5a2b62b3ea6cce0c0c02092b0b46ac6cb3b45fabb46dca17ad2a470336d61: Status 404 returned error can't find the container with id 2be5a2b62b3ea6cce0c0c02092b0b46ac6cb3b45fabb46dca17ad2a470336d61 Jul 22 16:53:38.465209 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:38.465175 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-689b965f47-4wx72" event={"ID":"fc927414-66ea-4c2a-a716-4be30de05f17","Type":"ContainerStarted","Data":"6cf9da32b76fc5f8846feeb560215ba673e87fb9b517a843a8b7829ddd84a6c2"} Jul 22 16:53:38.465209 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:38.465215 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-689b965f47-4wx72" event={"ID":"fc927414-66ea-4c2a-a716-4be30de05f17","Type":"ContainerStarted","Data":"2be5a2b62b3ea6cce0c0c02092b0b46ac6cb3b45fabb46dca17ad2a470336d61"} Jul 22 16:53:38.594201 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:38.594134 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-689b965f47-4wx72" podStartSLOduration=1.241665154 podStartE2EDuration="1.594110513s" totalImagesPullingTime="352.445359ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:53:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:53:38.520751998 +0000 UTC m=+805.431409171" watchObservedRunningTime="2026-07-22 16:53:38.594110513 +0000 UTC m=+805.504767686" Jul 22 16:53:38.596847 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:38.596815 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fdf59ddf8-9ltqb"] Jul 22 16:53:38.597062 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:38.597035 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" podUID="845c8797-4295-4a10-b121-3f0fe365d977" containerName="authorino" containerID="cri-o://d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1" gracePeriod=30 Jul 22 16:53:38.859863 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:38.859838 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" Jul 22 16:53:38.910492 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:38.910457 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/845c8797-4295-4a10-b121-3f0fe365d977-tls-cert\") pod \"845c8797-4295-4a10-b121-3f0fe365d977\" (UID: \"845c8797-4295-4a10-b121-3f0fe365d977\") " Jul 22 16:53:38.910678 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:38.910525 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd2n9\" (UniqueName: \"kubernetes.io/projected/845c8797-4295-4a10-b121-3f0fe365d977-kube-api-access-jd2n9\") pod \"845c8797-4295-4a10-b121-3f0fe365d977\" (UID: \"845c8797-4295-4a10-b121-3f0fe365d977\") " Jul 22 16:53:38.912990 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:38.912955 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845c8797-4295-4a10-b121-3f0fe365d977-kube-api-access-jd2n9" (OuterVolumeSpecName: "kube-api-access-jd2n9") pod "845c8797-4295-4a10-b121-3f0fe365d977" (UID: "845c8797-4295-4a10-b121-3f0fe365d977"). InnerVolumeSpecName "kube-api-access-jd2n9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:53:38.922390 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:38.922336 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/845c8797-4295-4a10-b121-3f0fe365d977-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "845c8797-4295-4a10-b121-3f0fe365d977" (UID: "845c8797-4295-4a10-b121-3f0fe365d977"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:53:39.012000 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.011955 2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/845c8797-4295-4a10-b121-3f0fe365d977-tls-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:53:39.012000 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.012002 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd2n9\" (UniqueName: \"kubernetes.io/projected/845c8797-4295-4a10-b121-3f0fe365d977-kube-api-access-jd2n9\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 16:53:39.472304 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.472259 2581 generic.go:356] "Generic (PLEG): container finished" podID="845c8797-4295-4a10-b121-3f0fe365d977" containerID="d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1" exitCode=0 Jul 22 16:53:39.472735 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.472347 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" Jul 22 16:53:39.472735 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.472389 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" event={"ID":"845c8797-4295-4a10-b121-3f0fe365d977","Type":"ContainerDied","Data":"d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1"} Jul 22 16:53:39.472735 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.472437 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdf59ddf8-9ltqb" event={"ID":"845c8797-4295-4a10-b121-3f0fe365d977","Type":"ContainerDied","Data":"94d98ddeced8ff5ec9c81e9313023171fe3226f019b022a779f37f6e707756e7"} Jul 22 16:53:39.472735 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.472456 2581 scope.go:117] "RemoveContainer" containerID="d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1" Jul 22 16:53:39.482498 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.482476 2581 scope.go:117] "RemoveContainer" containerID="d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1" Jul 22 16:53:39.482813 ip-10-0-132-57 kubenswrapper[2581]: E0722 16:53:39.482793 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1\": container with ID starting with d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1 not found: ID does not exist" containerID="d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1" Jul 22 16:53:39.482880 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.482821 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1"} err="failed to get container status \"d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1\": rpc error: code = NotFound desc = could not find container \"d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1\": container with ID starting with d4df4a896c579cdbde9f6139abf72aced3617087d94c1dd0795e201158b3ebd1 not found: ID does not exist" Jul 22 16:53:39.500669 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.500642 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fdf59ddf8-9ltqb"] Jul 22 16:53:39.502913 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.502891 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7fdf59ddf8-9ltqb"] Jul 22 16:53:39.618347 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:53:39.618316 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="845c8797-4295-4a10-b121-3f0fe365d977" path="/var/lib/kubelet/pods/845c8797-4295-4a10-b121-3f0fe365d977/volumes" Jul 22 16:55:13.608156 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:55:13.608074 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 16:55:13.611166 ip-10-0-132-57 kubenswrapper[2581]: I0722 16:55:13.611142 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:00:13.641908 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:00:13.641879 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:00:13.647971 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:00:13.647943 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:05:13.678023 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:05:13.677997 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:05:13.687971 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:05:13.687947 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:10:13.715650 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:10:13.715625 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:10:13.724857 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:10:13.724838 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:15:13.750766 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:15:13.750740 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:15:13.759611 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:15:13.759588 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:17:50.510827 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.510789 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r"] Jul 22 17:17:50.511406 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.511304 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="845c8797-4295-4a10-b121-3f0fe365d977" containerName="authorino" Jul 22 17:17:50.511406 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.511325 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="845c8797-4295-4a10-b121-3f0fe365d977" containerName="authorino" Jul 22 17:17:50.511550 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.511479 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="845c8797-4295-4a10-b121-3f0fe365d977" containerName="authorino" Jul 22 17:17:50.514436 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.514416 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.522416 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.517410 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-668a76\"/\"openshift-service-ca.crt\"" Jul 22 17:17:50.522416 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.517710 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-668a76\"/\"kube-root-ca.crt\"" Jul 22 17:17:50.522416 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.518245 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-668a76\"/\"test-model-668a76-kserve-self-signed-certs\"" Jul 22 17:17:50.522416 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.518520 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-668a76\"/\"default-dockercfg-rs5tk\"" Jul 22 17:17:50.524599 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.524577 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r"] Jul 22 17:17:50.613298 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.613269 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-model-cache\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.613447 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.613303 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx7sb\" (UniqueName: \"kubernetes.io/projected/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-kube-api-access-bx7sb\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.613447 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.613361 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tls-certs\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.613447 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.613432 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-home\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.613606 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.613470 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-dshm\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.613606 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.613517 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tmp-dir\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.714467 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.714444 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-home\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.714589 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.714483 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-dshm\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.714589 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.714506 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tmp-dir\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.714589 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.714534 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-model-cache\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.714762 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.714642 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7sb\" (UniqueName: \"kubernetes.io/projected/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-kube-api-access-bx7sb\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.714762 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.714706 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tls-certs\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.714884 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.714864 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-model-cache\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.714943 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.714917 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-home\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.715223 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.715202 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tmp-dir\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.716657 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.716640 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-dshm\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.717104 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.717086 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tls-certs\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.722399 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.722380 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx7sb\" (UniqueName: \"kubernetes.io/projected/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-kube-api-access-bx7sb\") pod \"test-model-668a76-kserve-6c74988b69-drg7r\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.827384 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.827310 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:17:50.958724 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.958697 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r"] Jul 22 17:17:50.960514 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:17:50.960485 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf77c596f_8ca4_4d1e_9ae3_196a5a7368c8.slice/crio-2aa52767147aee36ab2d410e0a2ae5776897a1209724ac741b3aa0aa4fff07bb WatchSource:0}: Error finding container 2aa52767147aee36ab2d410e0a2ae5776897a1209724ac741b3aa0aa4fff07bb: Status 404 returned error can't find the container with id 2aa52767147aee36ab2d410e0a2ae5776897a1209724ac741b3aa0aa4fff07bb Jul 22 17:17:50.983093 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:50.983067 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" event={"ID":"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8","Type":"ContainerStarted","Data":"2aa52767147aee36ab2d410e0a2ae5776897a1209724ac741b3aa0aa4fff07bb"} Jul 22 17:17:51.989429 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:17:51.989395 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" event={"ID":"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8","Type":"ContainerStarted","Data":"60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d"} Jul 22 17:18:00.828006 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:00.827973 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:18:00.828526 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:00.828016 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:18:00.840178 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:00.840155 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:18:00.858411 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:00.858354 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" podStartSLOduration=10.858339251 podStartE2EDuration="10.858339251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:17:52.006384951 +0000 UTC m=+2258.917042133" watchObservedRunningTime="2026-07-22 17:18:00.858339251 +0000 UTC m=+2267.768996423" Jul 22 17:18:01.045947 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:01.045919 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:18:15.298196 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.298161 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv"] Jul 22 17:18:15.302784 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.302765 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.305156 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.305134 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f854fd\"/\"default-dockercfg-lvn8n\"" Jul 22 17:18:15.305261 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.305137 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f854fd\"/\"openshift-service-ca.crt\"" Jul 22 17:18:15.305261 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.305141 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f854fd\"/\"kube-root-ca.crt\"" Jul 22 17:18:15.306067 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.306048 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f854fd\"/\"test-model-f854fd-kserve-self-signed-certs\"" Jul 22 17:18:15.310009 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.309980 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv"] Jul 22 17:18:15.403790 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.403766 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-model-cache\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.403904 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.403797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-dshm\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.403904 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.403815 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57391756-0be0-4c94-8bd5-a39bc4f407cc-tls-certs\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.403904 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.403839 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-home\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.403904 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.403887 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s74vs\" (UniqueName: \"kubernetes.io/projected/57391756-0be0-4c94-8bd5-a39bc4f407cc-kube-api-access-s74vs\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.404056 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.403911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-tmp-dir\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.504258 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.504228 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-dshm\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.504258 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.504259 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57391756-0be0-4c94-8bd5-a39bc4f407cc-tls-certs\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.504482 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.504283 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-home\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.504482 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.504309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s74vs\" (UniqueName: \"kubernetes.io/projected/57391756-0be0-4c94-8bd5-a39bc4f407cc-kube-api-access-s74vs\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.504482 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.504333 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-tmp-dir\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.504482 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.504414 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-model-cache\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.504743 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.504720 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-home\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.504800 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.504751 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-tmp-dir\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.504837 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.504799 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-model-cache\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.506632 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.506612 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-dshm\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.506791 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.506775 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57391756-0be0-4c94-8bd5-a39bc4f407cc-tls-certs\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.512263 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.512243 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s74vs\" (UniqueName: \"kubernetes.io/projected/57391756-0be0-4c94-8bd5-a39bc4f407cc-kube-api-access-s74vs\") pod \"test-model-f854fd-kserve-6587bbbcff-4xjpv\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.613168 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.613110 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:15.744383 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:15.744221 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv"] Jul 22 17:18:15.746981 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:18:15.746957 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57391756_0be0_4c94_8bd5_a39bc4f407cc.slice/crio-47c05cfa316c275001f9a139831c03c043dee584b58ce69b26ff125fa32804b7 WatchSource:0}: Error finding container 47c05cfa316c275001f9a139831c03c043dee584b58ce69b26ff125fa32804b7: Status 404 returned error can't find the container with id 47c05cfa316c275001f9a139831c03c043dee584b58ce69b26ff125fa32804b7 Jul 22 17:18:16.112211 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:16.112175 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" event={"ID":"57391756-0be0-4c94-8bd5-a39bc4f407cc","Type":"ContainerStarted","Data":"a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f"} Jul 22 17:18:16.112211 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:16.112215 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" event={"ID":"57391756-0be0-4c94-8bd5-a39bc4f407cc","Type":"ContainerStarted","Data":"47c05cfa316c275001f9a139831c03c043dee584b58ce69b26ff125fa32804b7"} Jul 22 17:18:16.131279 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:16.131234 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" podStartSLOduration=1.131218212 podStartE2EDuration="1.131218212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:18:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:18:16.129496658 +0000 UTC m=+2283.040153829" watchObservedRunningTime="2026-07-22 17:18:16.131218212 +0000 UTC m=+2283.041875384" Jul 22 17:18:25.618862 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:25.618829 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:25.618862 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:25.618859 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:25.627615 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:25.627592 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:26.183158 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:26.183130 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:18:53.575459 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:53.575417 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r"] Jul 22 17:18:54.615093 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:18:54.615059 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv"] Jul 22 17:19:08.595333 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.595228 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" podUID="f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" containerName="main" containerID="cri-o://60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d" gracePeriod=45 Jul 22 17:19:08.854338 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.854284 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:19:08.944174 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944144 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-model-cache\") pod \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " Jul 22 17:19:08.944319 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944179 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tmp-dir\") pod \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " Jul 22 17:19:08.944319 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944209 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-dshm\") pod \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " Jul 22 17:19:08.944319 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944270 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx7sb\" (UniqueName: \"kubernetes.io/projected/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-kube-api-access-bx7sb\") pod \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " Jul 22 17:19:08.944319 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944292 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-home\") pod \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " Jul 22 17:19:08.944631 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944347 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tls-certs\") pod \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\" (UID: \"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8\") " Jul 22 17:19:08.944631 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944384 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-model-cache" (OuterVolumeSpecName: "model-cache") pod "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" (UID: "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:19:08.944631 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944412 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" (UID: "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:19:08.944631 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944611 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-home" (OuterVolumeSpecName: "home") pod "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" (UID: "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:19:08.944786 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944688 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-model-cache\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:08.944786 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.944707 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tmp-dir\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:08.946544 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.946512 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" (UID: "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 17:19:08.946644 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.946549 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-kube-api-access-bx7sb" (OuterVolumeSpecName: "kube-api-access-bx7sb") pod "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" (UID: "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8"). InnerVolumeSpecName "kube-api-access-bx7sb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:19:08.946644 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:08.946583 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-dshm" (OuterVolumeSpecName: "dshm") pod "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" (UID: "f77c596f-8ca4-4d1e-9ae3-196a5a7368c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:19:09.045721 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.045699 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-dshm\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:09.045721 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.045720 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx7sb\" (UniqueName: \"kubernetes.io/projected/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-kube-api-access-bx7sb\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:09.045871 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.045730 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-home\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:09.045871 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.045740 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8-tls-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:09.401238 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.401206 2581 generic.go:356] "Generic (PLEG): container finished" podID="f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" containerID="60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d" exitCode=0 Jul 22 17:19:09.401428 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.401253 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" Jul 22 17:19:09.401428 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.401258 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" event={"ID":"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8","Type":"ContainerDied","Data":"60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d"} Jul 22 17:19:09.401428 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.401381 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r" event={"ID":"f77c596f-8ca4-4d1e-9ae3-196a5a7368c8","Type":"ContainerDied","Data":"2aa52767147aee36ab2d410e0a2ae5776897a1209724ac741b3aa0aa4fff07bb"} Jul 22 17:19:09.401428 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.401420 2581 scope.go:117] "RemoveContainer" containerID="60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d" Jul 22 17:19:09.413439 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.413417 2581 scope.go:117] "RemoveContainer" containerID="60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d" Jul 22 17:19:09.413692 ip-10-0-132-57 kubenswrapper[2581]: E0722 17:19:09.413670 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d\": container with ID starting with 60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d not found: ID does not exist" containerID="60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d" Jul 22 17:19:09.413741 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.413701 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d"} err="failed to get container status \"60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d\": rpc error: code = NotFound desc = could not find container \"60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d\": container with ID starting with 60125251793dad5dd8bc6ee28273f27b7a47dc22b2225c296cc351eb3fadda1d not found: ID does not exist" Jul 22 17:19:09.428988 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.428964 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r"] Jul 22 17:19:09.433125 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.433103 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-668a76/test-model-668a76-kserve-6c74988b69-drg7r"] Jul 22 17:19:09.618768 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.618734 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" path="/var/lib/kubelet/pods/f77c596f-8ca4-4d1e-9ae3-196a5a7368c8/volumes" Jul 22 17:19:09.635652 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.635620 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" podUID="57391756-0be0-4c94-8bd5-a39bc4f407cc" containerName="main" containerID="cri-o://a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f" gracePeriod=45 Jul 22 17:19:09.875533 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.875510 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:19:09.953132 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953106 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-tmp-dir\") pod \"57391756-0be0-4c94-8bd5-a39bc4f407cc\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " Jul 22 17:19:09.953252 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953138 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-model-cache\") pod \"57391756-0be0-4c94-8bd5-a39bc4f407cc\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " Jul 22 17:19:09.953252 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953187 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s74vs\" (UniqueName: \"kubernetes.io/projected/57391756-0be0-4c94-8bd5-a39bc4f407cc-kube-api-access-s74vs\") pod \"57391756-0be0-4c94-8bd5-a39bc4f407cc\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " Jul 22 17:19:09.953252 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953213 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-home\") pod \"57391756-0be0-4c94-8bd5-a39bc4f407cc\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " Jul 22 17:19:09.953252 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953234 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-dshm\") pod \"57391756-0be0-4c94-8bd5-a39bc4f407cc\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " Jul 22 17:19:09.953498 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953333 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57391756-0be0-4c94-8bd5-a39bc4f407cc-tls-certs\") pod \"57391756-0be0-4c94-8bd5-a39bc4f407cc\" (UID: \"57391756-0be0-4c94-8bd5-a39bc4f407cc\") " Jul 22 17:19:09.953498 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953353 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "57391756-0be0-4c94-8bd5-a39bc4f407cc" (UID: "57391756-0be0-4c94-8bd5-a39bc4f407cc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:19:09.953598 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953503 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-model-cache" (OuterVolumeSpecName: "model-cache") pod "57391756-0be0-4c94-8bd5-a39bc4f407cc" (UID: "57391756-0be0-4c94-8bd5-a39bc4f407cc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:19:09.953598 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953516 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-home" (OuterVolumeSpecName: "home") pod "57391756-0be0-4c94-8bd5-a39bc4f407cc" (UID: "57391756-0be0-4c94-8bd5-a39bc4f407cc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:19:09.953697 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953626 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-home\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:09.953697 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953646 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-tmp-dir\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:09.953697 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.953659 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-model-cache\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:09.955483 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.955453 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57391756-0be0-4c94-8bd5-a39bc4f407cc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "57391756-0be0-4c94-8bd5-a39bc4f407cc" (UID: "57391756-0be0-4c94-8bd5-a39bc4f407cc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 17:19:09.955483 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.955467 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-dshm" (OuterVolumeSpecName: "dshm") pod "57391756-0be0-4c94-8bd5-a39bc4f407cc" (UID: "57391756-0be0-4c94-8bd5-a39bc4f407cc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:19:09.955620 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:09.955488 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57391756-0be0-4c94-8bd5-a39bc4f407cc-kube-api-access-s74vs" (OuterVolumeSpecName: "kube-api-access-s74vs") pod "57391756-0be0-4c94-8bd5-a39bc4f407cc" (UID: "57391756-0be0-4c94-8bd5-a39bc4f407cc"). InnerVolumeSpecName "kube-api-access-s74vs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:19:10.054533 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.054480 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57391756-0be0-4c94-8bd5-a39bc4f407cc-tls-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:10.054533 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.054499 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s74vs\" (UniqueName: \"kubernetes.io/projected/57391756-0be0-4c94-8bd5-a39bc4f407cc-kube-api-access-s74vs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:10.054533 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.054508 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57391756-0be0-4c94-8bd5-a39bc4f407cc-dshm\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:19:10.407346 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.407255 2581 generic.go:356] "Generic (PLEG): container finished" podID="57391756-0be0-4c94-8bd5-a39bc4f407cc" containerID="a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f" exitCode=0 Jul 22 17:19:10.407346 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.407299 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" Jul 22 17:19:10.407585 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.407341 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" event={"ID":"57391756-0be0-4c94-8bd5-a39bc4f407cc","Type":"ContainerDied","Data":"a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f"} Jul 22 17:19:10.407585 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.407392 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv" event={"ID":"57391756-0be0-4c94-8bd5-a39bc4f407cc","Type":"ContainerDied","Data":"47c05cfa316c275001f9a139831c03c043dee584b58ce69b26ff125fa32804b7"} Jul 22 17:19:10.407585 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.407421 2581 scope.go:117] "RemoveContainer" containerID="a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f" Jul 22 17:19:10.416653 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.416623 2581 scope.go:117] "RemoveContainer" containerID="a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f" Jul 22 17:19:10.416905 ip-10-0-132-57 kubenswrapper[2581]: E0722 17:19:10.416886 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f\": container with ID starting with a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f not found: ID does not exist" containerID="a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f" Jul 22 17:19:10.416959 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.416912 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f"} err="failed to get container status \"a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f\": rpc error: code = NotFound desc = could not find container \"a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f\": container with ID starting with a6a565ac72402d0f58655295fc5f4550a016f10943efeecd0169e0f6f585413f not found: ID does not exist" Jul 22 17:19:10.439624 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.439602 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv"] Jul 22 17:19:10.446958 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:10.446923 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f854fd/test-model-f854fd-kserve-6587bbbcff-4xjpv"] Jul 22 17:19:11.618892 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:19:11.618857 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57391756-0be0-4c94-8bd5-a39bc4f407cc" path="/var/lib/kubelet/pods/57391756-0be0-4c94-8bd5-a39bc4f407cc/volumes" Jul 22 17:20:13.785183 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:20:13.785155 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:20:13.794387 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:20:13.794343 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:21:32.727502 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.727469 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5"] Jul 22 17:21:32.727942 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.727827 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" containerName="main" Jul 22 17:21:32.727942 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.727839 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" containerName="main" Jul 22 17:21:32.727942 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.727858 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57391756-0be0-4c94-8bd5-a39bc4f407cc" containerName="main" Jul 22 17:21:32.727942 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.727863 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="57391756-0be0-4c94-8bd5-a39bc4f407cc" containerName="main" Jul 22 17:21:32.727942 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.727925 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f77c596f-8ca4-4d1e-9ae3-196a5a7368c8" containerName="main" Jul 22 17:21:32.727942 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.727936 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="57391756-0be0-4c94-8bd5-a39bc4f407cc" containerName="main" Jul 22 17:21:32.731045 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.731027 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.733597 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.733576 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-629c13\"/\"auth-test-model-629c13-kserve-self-signed-certs\"" Jul 22 17:21:32.734919 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.734902 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-629c13\"/\"openshift-service-ca.crt\"" Jul 22 17:21:32.735064 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.735044 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-629c13\"/\"default-dockercfg-xqr92\"" Jul 22 17:21:32.735152 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.735088 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-629c13\"/\"kube-root-ca.crt\"" Jul 22 17:21:32.746830 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.746806 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5"] Jul 22 17:21:32.797704 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.797675 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-home\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.797856 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.797745 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-dshm\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.797856 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.797798 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtrdt\" (UniqueName: \"kubernetes.io/projected/7a7a58d5-723d-49e2-95f4-80d30dd25930-kube-api-access-jtrdt\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.797856 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.797823 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-model-cache\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.797856 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.797845 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a7a58d5-723d-49e2-95f4-80d30dd25930-tls-certs\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.798002 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.797915 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-tmp-dir\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.904663 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.899508 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-tmp-dir\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.904663 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.899586 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-home\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.904663 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.899679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-dshm\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.904663 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.899739 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtrdt\" (UniqueName: \"kubernetes.io/projected/7a7a58d5-723d-49e2-95f4-80d30dd25930-kube-api-access-jtrdt\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.904663 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.899790 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-model-cache\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.904663 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.899828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a7a58d5-723d-49e2-95f4-80d30dd25930-tls-certs\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.904663 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.902969 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a7a58d5-723d-49e2-95f4-80d30dd25930-tls-certs\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.904663 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.903274 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-tmp-dir\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.904663 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.903525 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-home\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.905868 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.905836 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-dshm\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.905990 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.905899 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-model-cache\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:32.914804 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:32.914783 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtrdt\" (UniqueName: \"kubernetes.io/projected/7a7a58d5-723d-49e2-95f4-80d30dd25930-kube-api-access-jtrdt\") pod \"auth-test-model-629c13-kserve-676b97fc87-55ms5\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:33.040697 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:33.040628 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:33.179049 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:33.179024 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5"] Jul 22 17:21:33.180890 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:21:33.180865 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a7a58d5_723d_49e2_95f4_80d30dd25930.slice/crio-4b35e7f5c60f0323112ee65b1293d14c2f61dddc442c84a1cb03c8661f97a471 WatchSource:0}: Error finding container 4b35e7f5c60f0323112ee65b1293d14c2f61dddc442c84a1cb03c8661f97a471: Status 404 returned error can't find the container with id 4b35e7f5c60f0323112ee65b1293d14c2f61dddc442c84a1cb03c8661f97a471 Jul 22 17:21:34.154595 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:34.154558 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" event={"ID":"7a7a58d5-723d-49e2-95f4-80d30dd25930","Type":"ContainerStarted","Data":"2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a"} Jul 22 17:21:34.154988 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:34.154606 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" event={"ID":"7a7a58d5-723d-49e2-95f4-80d30dd25930","Type":"ContainerStarted","Data":"4b35e7f5c60f0323112ee65b1293d14c2f61dddc442c84a1cb03c8661f97a471"} Jul 22 17:21:43.041009 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:43.040970 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:43.041403 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:43.041027 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:43.053143 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:43.053119 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:21:43.071061 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:43.071013 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" podStartSLOduration=11.070999366 podStartE2EDuration="11.070999366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:21:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:21:34.170248462 +0000 UTC m=+2481.080905635" watchObservedRunningTime="2026-07-22 17:21:43.070999366 +0000 UTC m=+2489.981656538" Jul 22 17:21:43.215887 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:21:43.215855 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:24:18.540935 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:18.540903 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5"] Jul 22 17:24:27.501207 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.501171 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj"] Jul 22 17:24:27.504634 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.504617 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.509895 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.509866 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-629c13\"/\"sub-test-model-629c13-kserve-self-signed-certs\"" Jul 22 17:24:27.516409 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.516382 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj"] Jul 22 17:24:27.691866 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.691834 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-dshm\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.691866 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.691867 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tmp-dir\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.692082 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.691905 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvss4\" (UniqueName: \"kubernetes.io/projected/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-kube-api-access-dvss4\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.692082 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.691979 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tls-certs\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.692082 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.692021 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-model-cache\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.692082 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.692048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-home\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.793152 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.793084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tls-certs\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.793152 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.793119 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-model-cache\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.793334 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.793154 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-home\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.793334 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.793210 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-dshm\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.793334 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.793232 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tmp-dir\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.793334 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.793286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvss4\" (UniqueName: \"kubernetes.io/projected/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-kube-api-access-dvss4\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.793592 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.793562 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-model-cache\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.793654 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.793594 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-home\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.793745 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.793725 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tmp-dir\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.795399 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.795354 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-dshm\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.795694 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.795675 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tls-certs\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.802082 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.802064 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvss4\" (UniqueName: \"kubernetes.io/projected/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-kube-api-access-dvss4\") pod \"sub-test-model-629c13-kserve-55f7f4dd7-d95vj\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.815633 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.815608 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:27.948144 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:27.948116 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj"] Jul 22 17:24:27.949740 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:24:27.949711 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ee09369_a9a7_46cb_8a1c_28e8ab78c48b.slice/crio-38b49ca279c0618300f69273d576d23ed022d6c2520b90cd4ae30132d7e7d8e8 WatchSource:0}: Error finding container 38b49ca279c0618300f69273d576d23ed022d6c2520b90cd4ae30132d7e7d8e8: Status 404 returned error can't find the container with id 38b49ca279c0618300f69273d576d23ed022d6c2520b90cd4ae30132d7e7d8e8 Jul 22 17:24:28.067063 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:28.066978 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" event={"ID":"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b","Type":"ContainerStarted","Data":"8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14"} Jul 22 17:24:28.067063 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:28.067023 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" event={"ID":"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b","Type":"ContainerStarted","Data":"38b49ca279c0618300f69273d576d23ed022d6c2520b90cd4ae30132d7e7d8e8"} Jul 22 17:24:28.085498 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:28.085450 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" podStartSLOduration=1.085433782 podStartE2EDuration="1.085433782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:24:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:24:28.08305361 +0000 UTC m=+2654.993710794" watchObservedRunningTime="2026-07-22 17:24:28.085433782 +0000 UTC m=+2654.996090953" Jul 22 17:24:33.560676 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.560639 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" podUID="7a7a58d5-723d-49e2-95f4-80d30dd25930" containerName="main" containerID="cri-o://2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a" gracePeriod=45 Jul 22 17:24:33.805660 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.805634 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:24:33.838251 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838188 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-tmp-dir\") pod \"7a7a58d5-723d-49e2-95f4-80d30dd25930\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " Jul 22 17:24:33.838392 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838265 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a7a58d5-723d-49e2-95f4-80d30dd25930-tls-certs\") pod \"7a7a58d5-723d-49e2-95f4-80d30dd25930\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " Jul 22 17:24:33.838392 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838319 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtrdt\" (UniqueName: \"kubernetes.io/projected/7a7a58d5-723d-49e2-95f4-80d30dd25930-kube-api-access-jtrdt\") pod \"7a7a58d5-723d-49e2-95f4-80d30dd25930\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " Jul 22 17:24:33.838392 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838349 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-dshm\") pod \"7a7a58d5-723d-49e2-95f4-80d30dd25930\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " Jul 22 17:24:33.838392 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838389 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-model-cache\") pod \"7a7a58d5-723d-49e2-95f4-80d30dd25930\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " Jul 22 17:24:33.838646 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838444 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a7a58d5-723d-49e2-95f4-80d30dd25930" (UID: "7a7a58d5-723d-49e2-95f4-80d30dd25930"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:24:33.838646 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838580 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-home\") pod \"7a7a58d5-723d-49e2-95f4-80d30dd25930\" (UID: \"7a7a58d5-723d-49e2-95f4-80d30dd25930\") " Jul 22 17:24:33.838774 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838650 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a7a58d5-723d-49e2-95f4-80d30dd25930" (UID: "7a7a58d5-723d-49e2-95f4-80d30dd25930"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:24:33.838930 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838901 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-tmp-dir\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:24:33.838930 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838917 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-home" (OuterVolumeSpecName: "home") pod "7a7a58d5-723d-49e2-95f4-80d30dd25930" (UID: "7a7a58d5-723d-49e2-95f4-80d30dd25930"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:24:33.839072 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.838922 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-model-cache\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:24:33.840589 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.840561 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-dshm" (OuterVolumeSpecName: "dshm") pod "7a7a58d5-723d-49e2-95f4-80d30dd25930" (UID: "7a7a58d5-723d-49e2-95f4-80d30dd25930"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:24:33.840682 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.840629 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a7a58d5-723d-49e2-95f4-80d30dd25930-kube-api-access-jtrdt" (OuterVolumeSpecName: "kube-api-access-jtrdt") pod "7a7a58d5-723d-49e2-95f4-80d30dd25930" (UID: "7a7a58d5-723d-49e2-95f4-80d30dd25930"). InnerVolumeSpecName "kube-api-access-jtrdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:24:33.840849 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.840829 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a7a58d5-723d-49e2-95f4-80d30dd25930-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a7a58d5-723d-49e2-95f4-80d30dd25930" (UID: "7a7a58d5-723d-49e2-95f4-80d30dd25930"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 17:24:33.940107 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.940085 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-home\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:24:33.940107 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.940105 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a7a58d5-723d-49e2-95f4-80d30dd25930-tls-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:24:33.940234 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.940116 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtrdt\" (UniqueName: \"kubernetes.io/projected/7a7a58d5-723d-49e2-95f4-80d30dd25930-kube-api-access-jtrdt\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:24:33.940234 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:33.940124 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a7a58d5-723d-49e2-95f4-80d30dd25930-dshm\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:24:34.099889 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:34.099808 2581 generic.go:356] "Generic (PLEG): container finished" podID="7a7a58d5-723d-49e2-95f4-80d30dd25930" containerID="2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a" exitCode=0 Jul 22 17:24:34.099889 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:34.099852 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" Jul 22 17:24:34.100052 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:34.099894 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" event={"ID":"7a7a58d5-723d-49e2-95f4-80d30dd25930","Type":"ContainerDied","Data":"2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a"} Jul 22 17:24:34.100052 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:34.099945 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5" event={"ID":"7a7a58d5-723d-49e2-95f4-80d30dd25930","Type":"ContainerDied","Data":"4b35e7f5c60f0323112ee65b1293d14c2f61dddc442c84a1cb03c8661f97a471"} Jul 22 17:24:34.100052 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:34.099963 2581 scope.go:117] "RemoveContainer" containerID="2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a" Jul 22 17:24:34.110063 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:34.110047 2581 scope.go:117] "RemoveContainer" containerID="2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a" Jul 22 17:24:34.110292 ip-10-0-132-57 kubenswrapper[2581]: E0722 17:24:34.110274 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a\": container with ID starting with 2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a not found: ID does not exist" containerID="2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a" Jul 22 17:24:34.110342 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:34.110300 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a"} err="failed to get container status \"2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a\": rpc error: code = NotFound desc = could not find container \"2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a\": container with ID starting with 2f142e75333b086a13cf3e101b3ca46c77d7b192a1f97fcee1a44ba1d4ccc46a not found: ID does not exist" Jul 22 17:24:34.121818 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:34.121797 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5"] Jul 22 17:24:34.126996 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:34.126975 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-629c13/auth-test-model-629c13-kserve-676b97fc87-55ms5"] Jul 22 17:24:35.619143 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:35.619112 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a7a58d5-723d-49e2-95f4-80d30dd25930" path="/var/lib/kubelet/pods/7a7a58d5-723d-49e2-95f4-80d30dd25930/volumes" Jul 22 17:24:37.816182 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:37.816151 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:37.816580 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:37.816192 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:37.828689 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:37.828667 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:38.132243 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:38.132170 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:24:45.506052 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.506019 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl"] Jul 22 17:24:45.506457 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.506356 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a7a58d5-723d-49e2-95f4-80d30dd25930" containerName="main" Jul 22 17:24:45.506457 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.506384 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a7a58d5-723d-49e2-95f4-80d30dd25930" containerName="main" Jul 22 17:24:45.506543 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.506460 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a7a58d5-723d-49e2-95f4-80d30dd25930" containerName="main" Jul 22 17:24:45.510232 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.510216 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.513462 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.513435 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-05e61d\"/\"openshift-service-ca.crt\"" Jul 22 17:24:45.513462 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.513435 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-05e61d\"/\"kube-root-ca.crt\"" Jul 22 17:24:45.513639 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.513435 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-05e61d\"/\"default-dockercfg-tw7rz\"" Jul 22 17:24:45.513639 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.513435 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-05e61d\"/\"sub-test-model-05e61d-kserve-self-signed-certs\"" Jul 22 17:24:45.520063 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.520041 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl"] Jul 22 17:24:45.533987 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.533961 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tls-certs\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.534112 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.534000 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-home\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.534112 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.534067 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kwwh\" (UniqueName: \"kubernetes.io/projected/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-kube-api-access-2kwwh\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.534112 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.534093 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-dshm\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.534261 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.534155 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-model-cache\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.534261 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.534202 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tmp-dir\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.635269 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.635247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tmp-dir\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.635396 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.635302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tls-certs\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.635582 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.635561 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-home\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.635671 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.635651 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kwwh\" (UniqueName: \"kubernetes.io/projected/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-kube-api-access-2kwwh\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.635891 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.635706 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tmp-dir\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.635891 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.635885 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-home\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.636033 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.635895 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-dshm\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.636033 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.635969 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-model-cache\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.636306 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.636286 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-model-cache\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.637922 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.637890 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-dshm\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.638073 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.638054 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tls-certs\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.643832 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.643814 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kwwh\" (UniqueName: \"kubernetes.io/projected/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-kube-api-access-2kwwh\") pod \"sub-test-model-05e61d-kserve-d576dbb9c-hwjrl\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.820176 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.820102 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:45.956090 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:45.956063 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl"] Jul 22 17:24:45.957892 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:24:45.957867 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod672cded6_a2f7_40be_ad5a_b7fbb3ea0bad.slice/crio-8d1d28dd6a424c61e46a7bf27cae52b9ea14294ed931df37768552ecccf81548 WatchSource:0}: Error finding container 8d1d28dd6a424c61e46a7bf27cae52b9ea14294ed931df37768552ecccf81548: Status 404 returned error can't find the container with id 8d1d28dd6a424c61e46a7bf27cae52b9ea14294ed931df37768552ecccf81548 Jul 22 17:24:46.163314 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:46.163217 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" event={"ID":"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad","Type":"ContainerStarted","Data":"82e373f6d1a60136e7a4a4af7e6b5f4d31b14180d9542cd7166566caafb20273"} Jul 22 17:24:46.163314 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:46.163264 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" event={"ID":"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad","Type":"ContainerStarted","Data":"8d1d28dd6a424c61e46a7bf27cae52b9ea14294ed931df37768552ecccf81548"} Jul 22 17:24:46.182304 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:46.182252 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" podStartSLOduration=1.182237598 podStartE2EDuration="1.182237598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:24:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:24:46.180352619 +0000 UTC m=+2673.091009791" watchObservedRunningTime="2026-07-22 17:24:46.182237598 +0000 UTC m=+2673.092894769" Jul 22 17:24:55.820729 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:55.820697 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:55.821210 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:55.820741 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:55.833092 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:55.833068 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:24:56.234570 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:24:56.234543 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:25:13.826585 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:13.826514 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:25:13.834514 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:13.834489 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:25:25.374298 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:25.374265 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj"] Jul 22 17:25:26.325608 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:26.325574 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl"] Jul 22 17:25:40.395718 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.395676 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" podUID="9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" containerName="main" containerID="cri-o://8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14" gracePeriod=45 Jul 22 17:25:40.651043 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.650978 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:25:40.684885 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.684857 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvss4\" (UniqueName: \"kubernetes.io/projected/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-kube-api-access-dvss4\") pod \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " Jul 22 17:25:40.684997 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.684891 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-model-cache\") pod \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " Jul 22 17:25:40.684997 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.684909 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-home\") pod \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " Jul 22 17:25:40.684997 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.684987 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tmp-dir\") pod \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " Jul 22 17:25:40.685134 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.685031 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tls-certs\") pod \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " Jul 22 17:25:40.685134 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.685059 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-dshm\") pod \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\" (UID: \"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b\") " Jul 22 17:25:40.685134 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.685099 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-model-cache" (OuterVolumeSpecName: "model-cache") pod "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" (UID: "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:25:40.685291 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.685210 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-home" (OuterVolumeSpecName: "home") pod "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" (UID: "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:25:40.685345 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.685302 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" (UID: "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:25:40.685539 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.685523 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-model-cache\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:40.685612 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.685544 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-home\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:40.685612 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.685558 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tmp-dir\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:40.687160 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.687132 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" (UID: "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 17:25:40.687457 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.687434 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-dshm" (OuterVolumeSpecName: "dshm") pod "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" (UID: "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:25:40.687540 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.687476 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-kube-api-access-dvss4" (OuterVolumeSpecName: "kube-api-access-dvss4") pod "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" (UID: "9ee09369-a9a7-46cb-8a1c-28e8ab78c48b"). InnerVolumeSpecName "kube-api-access-dvss4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:25:40.786019 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.785989 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-tls-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:40.786019 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.786016 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-dshm\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:40.786179 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:40.786029 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvss4\" (UniqueName: \"kubernetes.io/projected/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b-kube-api-access-dvss4\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:41.345289 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.345252 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" podUID="672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" containerName="main" containerID="cri-o://82e373f6d1a60136e7a4a4af7e6b5f4d31b14180d9542cd7166566caafb20273" gracePeriod=45 Jul 22 17:25:41.479456 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.479431 2581 generic.go:356] "Generic (PLEG): container finished" podID="672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" containerID="82e373f6d1a60136e7a4a4af7e6b5f4d31b14180d9542cd7166566caafb20273" exitCode=0 Jul 22 17:25:41.479785 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.479505 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" event={"ID":"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad","Type":"ContainerDied","Data":"82e373f6d1a60136e7a4a4af7e6b5f4d31b14180d9542cd7166566caafb20273"} Jul 22 17:25:41.480930 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.480907 2581 generic.go:356] "Generic (PLEG): container finished" podID="9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" containerID="8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14" exitCode=0 Jul 22 17:25:41.481043 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.480948 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" event={"ID":"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b","Type":"ContainerDied","Data":"8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14"} Jul 22 17:25:41.481043 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.480951 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" Jul 22 17:25:41.481043 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.480970 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj" event={"ID":"9ee09369-a9a7-46cb-8a1c-28e8ab78c48b","Type":"ContainerDied","Data":"38b49ca279c0618300f69273d576d23ed022d6c2520b90cd4ae30132d7e7d8e8"} Jul 22 17:25:41.481043 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.480996 2581 scope.go:117] "RemoveContainer" containerID="8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14" Jul 22 17:25:41.491215 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.491194 2581 scope.go:117] "RemoveContainer" containerID="8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14" Jul 22 17:25:41.491496 ip-10-0-132-57 kubenswrapper[2581]: E0722 17:25:41.491468 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14\": container with ID starting with 8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14 not found: ID does not exist" containerID="8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14" Jul 22 17:25:41.491575 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.491495 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14"} err="failed to get container status \"8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14\": rpc error: code = NotFound desc = could not find container \"8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14\": container with ID starting with 8e32a42573cac29ec5ea5307d150db7cca8519f7b9f5f692366d043207653c14 not found: ID does not exist" Jul 22 17:25:41.502822 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.502799 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj"] Jul 22 17:25:41.506975 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.506954 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-629c13/sub-test-model-629c13-kserve-55f7f4dd7-d95vj"] Jul 22 17:25:41.602652 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.602598 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:25:41.619480 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.619458 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" path="/var/lib/kubelet/pods/9ee09369-a9a7-46cb-8a1c-28e8ab78c48b/volumes" Jul 22 17:25:41.693212 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.693188 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-dshm\") pod \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " Jul 22 17:25:41.693315 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.693241 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tls-certs\") pod \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " Jul 22 17:25:41.693315 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.693260 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-model-cache\") pod \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " Jul 22 17:25:41.693315 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.693281 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kwwh\" (UniqueName: \"kubernetes.io/projected/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-kube-api-access-2kwwh\") pod \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " Jul 22 17:25:41.693315 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.693306 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tmp-dir\") pod \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " Jul 22 17:25:41.693566 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.693352 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-home\") pod \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\" (UID: \"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad\") " Jul 22 17:25:41.693641 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.693615 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-model-cache" (OuterVolumeSpecName: "model-cache") pod "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" (UID: "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:25:41.693701 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.693635 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" (UID: "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:25:41.693750 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.693703 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-home" (OuterVolumeSpecName: "home") pod "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" (UID: "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:25:41.695344 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.695325 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" (UID: "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 17:25:41.695740 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.695718 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-kube-api-access-2kwwh" (OuterVolumeSpecName: "kube-api-access-2kwwh") pod "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" (UID: "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad"). InnerVolumeSpecName "kube-api-access-2kwwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:25:41.695828 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.695719 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-dshm" (OuterVolumeSpecName: "dshm") pod "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" (UID: "672cded6-a2f7-40be-ad5a-b7fbb3ea0bad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:25:41.794678 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.794651 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-home\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:41.794678 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.794673 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-dshm\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:41.794832 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.794682 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tls-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:41.794832 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.794698 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-model-cache\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:41.794832 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.794708 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kwwh\" (UniqueName: \"kubernetes.io/projected/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-kube-api-access-2kwwh\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:41.794832 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:41.794716 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad-tmp-dir\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:25:42.488351 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:42.488318 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" event={"ID":"672cded6-a2f7-40be-ad5a-b7fbb3ea0bad","Type":"ContainerDied","Data":"8d1d28dd6a424c61e46a7bf27cae52b9ea14294ed931df37768552ecccf81548"} Jul 22 17:25:42.488765 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:42.488385 2581 scope.go:117] "RemoveContainer" containerID="82e373f6d1a60136e7a4a4af7e6b5f4d31b14180d9542cd7166566caafb20273" Jul 22 17:25:42.488765 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:42.488326 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl" Jul 22 17:25:42.520919 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:42.520892 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl"] Jul 22 17:25:42.528245 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:42.528215 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/sub-test-model-05e61d-kserve-d576dbb9c-hwjrl"] Jul 22 17:25:43.624060 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:25:43.624029 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" path="/var/lib/kubelet/pods/672cded6-a2f7-40be-ad5a-b7fbb3ea0bad/volumes" Jul 22 17:26:20.199481 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.199447 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2"] Jul 22 17:26:20.200023 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.199986 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" containerName="main" Jul 22 17:26:20.200023 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.200006 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" containerName="main" Jul 22 17:26:20.200146 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.200026 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" containerName="main" Jul 22 17:26:20.200146 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.200034 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" containerName="main" Jul 22 17:26:20.200146 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.200136 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ee09369-a9a7-46cb-8a1c-28e8ab78c48b" containerName="main" Jul 22 17:26:20.200296 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.200148 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="672cded6-a2f7-40be-ad5a-b7fbb3ea0bad" containerName="main" Jul 22 17:26:20.204497 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.204475 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.207781 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.207759 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-05e61d\"/\"default-dockercfg-tw7rz\"" Jul 22 17:26:20.207903 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.207798 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-05e61d\"/\"kube-root-ca.crt\"" Jul 22 17:26:20.207903 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.207798 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-05e61d\"/\"openshift-service-ca.crt\"" Jul 22 17:26:20.207903 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.207884 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-05e61d\"/\"rate-test-model-05e61d-kserve-self-signed-certs\"" Jul 22 17:26:20.214409 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.214390 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2"] Jul 22 17:26:20.285169 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.285137 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-tmp-dir\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.285272 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.285179 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-dshm\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.285272 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.285198 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a80bca02-f840-4798-8e23-88ab29dfa43f-tls-certs\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.285386 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.285304 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-model-cache\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.285386 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.285341 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-home\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.285474 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.285384 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz6c2\" (UniqueName: \"kubernetes.io/projected/a80bca02-f840-4798-8e23-88ab29dfa43f-kube-api-access-xz6c2\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.386219 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.386189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-model-cache\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.386329 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.386236 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-home\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.386329 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.386274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz6c2\" (UniqueName: \"kubernetes.io/projected/a80bca02-f840-4798-8e23-88ab29dfa43f-kube-api-access-xz6c2\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.386494 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.386415 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-tmp-dir\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.386494 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.386448 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-dshm\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.386494 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.386464 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a80bca02-f840-4798-8e23-88ab29dfa43f-tls-certs\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.386641 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.386568 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-model-cache\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.386641 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.386617 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-home\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.386771 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.386755 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-tmp-dir\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.388576 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.388553 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-dshm\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.388841 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.388826 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a80bca02-f840-4798-8e23-88ab29dfa43f-tls-certs\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.394504 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.394486 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz6c2\" (UniqueName: \"kubernetes.io/projected/a80bca02-f840-4798-8e23-88ab29dfa43f-kube-api-access-xz6c2\") pod \"rate-test-model-05e61d-kserve-5876bbf5f-zcnz2\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.514788 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.514771 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:20.647046 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.647023 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2"] Jul 22 17:26:20.649502 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:26:20.649474 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda80bca02_f840_4798_8e23_88ab29dfa43f.slice/crio-93a545ae7c7a9edfa955df741f9a5439aa8df863e12212e777a51a18b2343e1d WatchSource:0}: Error finding container 93a545ae7c7a9edfa955df741f9a5439aa8df863e12212e777a51a18b2343e1d: Status 404 returned error can't find the container with id 93a545ae7c7a9edfa955df741f9a5439aa8df863e12212e777a51a18b2343e1d Jul 22 17:26:20.696023 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:20.695989 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" event={"ID":"a80bca02-f840-4798-8e23-88ab29dfa43f","Type":"ContainerStarted","Data":"93a545ae7c7a9edfa955df741f9a5439aa8df863e12212e777a51a18b2343e1d"} Jul 22 17:26:21.703254 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:21.703217 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" event={"ID":"a80bca02-f840-4798-8e23-88ab29dfa43f","Type":"ContainerStarted","Data":"20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520"} Jul 22 17:26:21.722813 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:21.722771 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" podStartSLOduration=1.722757313 podStartE2EDuration="1.722757313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 17:26:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:26:21.720787223 +0000 UTC m=+2768.631444420" watchObservedRunningTime="2026-07-22 17:26:21.722757313 +0000 UTC m=+2768.633414486" Jul 22 17:26:30.515077 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:30.515045 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:30.515544 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:30.515088 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:30.527683 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:30.527660 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:26:30.760508 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:26:30.760480 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:27:06.219884 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:06.219854 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2"] Jul 22 17:27:07.225171 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.225139 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7542-140412979740464"] Jul 22 17:27:07.228563 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.228545 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979740464" Jul 22 17:27:07.230765 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.230740 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mx2jw\"" Jul 22 17:27:07.232649 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.232627 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979740464"] Jul 22 17:27:07.276762 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.276742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjqvk\" (UniqueName: \"kubernetes.io/projected/8367998f-34b6-4b2d-acab-9c54d2ece909-kube-api-access-fjqvk\") pod \"test-curl-7542-140412979740464\" (UID: \"8367998f-34b6-4b2d-acab-9c54d2ece909\") " pod="opendatahub/test-curl-7542-140412979740464" Jul 22 17:27:07.377244 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.377215 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjqvk\" (UniqueName: \"kubernetes.io/projected/8367998f-34b6-4b2d-acab-9c54d2ece909-kube-api-access-fjqvk\") pod \"test-curl-7542-140412979740464\" (UID: \"8367998f-34b6-4b2d-acab-9c54d2ece909\") " pod="opendatahub/test-curl-7542-140412979740464" Jul 22 17:27:07.384840 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.384820 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjqvk\" (UniqueName: \"kubernetes.io/projected/8367998f-34b6-4b2d-acab-9c54d2ece909-kube-api-access-fjqvk\") pod \"test-curl-7542-140412979740464\" (UID: \"8367998f-34b6-4b2d-acab-9c54d2ece909\") " pod="opendatahub/test-curl-7542-140412979740464" Jul 22 17:27:07.539723 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.539659 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979740464" Jul 22 17:27:07.670243 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.670221 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979740464"] Jul 22 17:27:07.671881 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:27:07.671856 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8367998f_34b6_4b2d_acab_9c54d2ece909.slice/crio-fc5d6a28364badc2f9eb155d8113b184bab6a639c61c9cd16071f1f49aa29dc6 WatchSource:0}: Error finding container fc5d6a28364badc2f9eb155d8113b184bab6a639c61c9cd16071f1f49aa29dc6: Status 404 returned error can't find the container with id fc5d6a28364badc2f9eb155d8113b184bab6a639c61c9cd16071f1f49aa29dc6 Jul 22 17:27:07.673225 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.673203 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 17:27:07.938885 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:07.938817 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979740464" event={"ID":"8367998f-34b6-4b2d-acab-9c54d2ece909","Type":"ContainerStarted","Data":"fc5d6a28364badc2f9eb155d8113b184bab6a639c61c9cd16071f1f49aa29dc6"} Jul 22 17:27:08.945076 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:08.945041 2581 generic.go:356] "Generic (PLEG): container finished" podID="8367998f-34b6-4b2d-acab-9c54d2ece909" containerID="3cf2e4fc694bf6f801f3932ea32a289147cfa9ff15a7684513e795012a503303" exitCode=0 Jul 22 17:27:08.945630 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:08.945121 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979740464" event={"ID":"8367998f-34b6-4b2d-acab-9c54d2ece909","Type":"ContainerDied","Data":"3cf2e4fc694bf6f801f3932ea32a289147cfa9ff15a7684513e795012a503303"} Jul 22 17:27:10.082028 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.082005 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979740464" Jul 22 17:27:10.119001 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.118975 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7542-140412979740464_8367998f-34b6-4b2d-acab-9c54d2ece909/test-curl-7542-140412979740464/0.log" Jul 22 17:27:10.143199 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.143168 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7542-140412979740464"] Jul 22 17:27:10.146994 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.146972 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7542-140412979740464"] Jul 22 17:27:10.198350 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.198325 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjqvk\" (UniqueName: \"kubernetes.io/projected/8367998f-34b6-4b2d-acab-9c54d2ece909-kube-api-access-fjqvk\") pod \"8367998f-34b6-4b2d-acab-9c54d2ece909\" (UID: \"8367998f-34b6-4b2d-acab-9c54d2ece909\") " Jul 22 17:27:10.200435 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.200385 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8367998f-34b6-4b2d-acab-9c54d2ece909-kube-api-access-fjqvk" (OuterVolumeSpecName: "kube-api-access-fjqvk") pod "8367998f-34b6-4b2d-acab-9c54d2ece909" (UID: "8367998f-34b6-4b2d-acab-9c54d2ece909"). InnerVolumeSpecName "kube-api-access-fjqvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:27:10.269504 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.269480 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7542-140412979739824"] Jul 22 17:27:10.269815 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.269804 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8367998f-34b6-4b2d-acab-9c54d2ece909" containerName="test-curl-7542-140412979740464" Jul 22 17:27:10.269859 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.269817 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8367998f-34b6-4b2d-acab-9c54d2ece909" containerName="test-curl-7542-140412979740464" Jul 22 17:27:10.269897 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.269884 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8367998f-34b6-4b2d-acab-9c54d2ece909" containerName="test-curl-7542-140412979740464" Jul 22 17:27:10.272848 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.272832 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979739824" Jul 22 17:27:10.279171 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.279148 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979739824"] Jul 22 17:27:10.299624 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.299600 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjqvk\" (UniqueName: \"kubernetes.io/projected/8367998f-34b6-4b2d-acab-9c54d2ece909-kube-api-access-fjqvk\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:10.400682 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.400661 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttv94\" (UniqueName: \"kubernetes.io/projected/8a7d2796-0f06-47f2-ba76-f6dc74313f33-kube-api-access-ttv94\") pod \"test-curl-7542-140412979739824\" (UID: \"8a7d2796-0f06-47f2-ba76-f6dc74313f33\") " pod="opendatahub/test-curl-7542-140412979739824" Jul 22 17:27:10.501618 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.501594 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttv94\" (UniqueName: \"kubernetes.io/projected/8a7d2796-0f06-47f2-ba76-f6dc74313f33-kube-api-access-ttv94\") pod \"test-curl-7542-140412979739824\" (UID: \"8a7d2796-0f06-47f2-ba76-f6dc74313f33\") " pod="opendatahub/test-curl-7542-140412979739824" Jul 22 17:27:10.509681 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.509657 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttv94\" (UniqueName: \"kubernetes.io/projected/8a7d2796-0f06-47f2-ba76-f6dc74313f33-kube-api-access-ttv94\") pod \"test-curl-7542-140412979739824\" (UID: \"8a7d2796-0f06-47f2-ba76-f6dc74313f33\") " pod="opendatahub/test-curl-7542-140412979739824" Jul 22 17:27:10.583305 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.583281 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979739824" Jul 22 17:27:10.723984 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.723959 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979739824"] Jul 22 17:27:10.725629 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:27:10.725603 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a7d2796_0f06_47f2_ba76_f6dc74313f33.slice/crio-4c5d4dbd93adc8263cbb0d6f78af9f62667a4764c1536f4593cdffbc41289228 WatchSource:0}: Error finding container 4c5d4dbd93adc8263cbb0d6f78af9f62667a4764c1536f4593cdffbc41289228: Status 404 returned error can't find the container with id 4c5d4dbd93adc8263cbb0d6f78af9f62667a4764c1536f4593cdffbc41289228 Jul 22 17:27:10.956539 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.956515 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979740464" Jul 22 17:27:10.956539 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.956530 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc5d6a28364badc2f9eb155d8113b184bab6a639c61c9cd16071f1f49aa29dc6" Jul 22 17:27:10.957826 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:10.957801 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979739824" event={"ID":"8a7d2796-0f06-47f2-ba76-f6dc74313f33","Type":"ContainerStarted","Data":"4c5d4dbd93adc8263cbb0d6f78af9f62667a4764c1536f4593cdffbc41289228"} Jul 22 17:27:10.967009 ip-10-0-132-57 kubenswrapper[2581]: E0722 17:27:10.966984 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7542-140412979740464\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="8367998f-34b6-4b2d-acab-9c54d2ece909" pod="opendatahub/test-curl-7542-140412979740464" Jul 22 17:27:11.618442 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:11.618410 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8367998f-34b6-4b2d-acab-9c54d2ece909" path="/var/lib/kubelet/pods/8367998f-34b6-4b2d-acab-9c54d2ece909/volumes" Jul 22 17:27:11.964402 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:11.964298 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a7d2796-0f06-47f2-ba76-f6dc74313f33" containerID="fc48019f5047e17e6277e60295e619a3cf32fb2b762e43e427b1638b37452e9c" exitCode=0 Jul 22 17:27:11.964402 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:11.964351 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979739824" event={"ID":"8a7d2796-0f06-47f2-ba76-f6dc74313f33","Type":"ContainerDied","Data":"fc48019f5047e17e6277e60295e619a3cf32fb2b762e43e427b1638b37452e9c"} Jul 22 17:27:13.096526 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.096504 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979739824" Jul 22 17:27:13.115453 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.115424 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7542-140412979739824_8a7d2796-0f06-47f2-ba76-f6dc74313f33/test-curl-7542-140412979739824/0.log" Jul 22 17:27:13.139313 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.139284 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7542-140412979739824"] Jul 22 17:27:13.142760 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.142737 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7542-140412979739824"] Jul 22 17:27:13.224136 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.224115 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttv94\" (UniqueName: \"kubernetes.io/projected/8a7d2796-0f06-47f2-ba76-f6dc74313f33-kube-api-access-ttv94\") pod \"8a7d2796-0f06-47f2-ba76-f6dc74313f33\" (UID: \"8a7d2796-0f06-47f2-ba76-f6dc74313f33\") " Jul 22 17:27:13.226110 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.226087 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7d2796-0f06-47f2-ba76-f6dc74313f33-kube-api-access-ttv94" (OuterVolumeSpecName: "kube-api-access-ttv94") pod "8a7d2796-0f06-47f2-ba76-f6dc74313f33" (UID: "8a7d2796-0f06-47f2-ba76-f6dc74313f33"). InnerVolumeSpecName "kube-api-access-ttv94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:27:13.301455 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.301422 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7542-140412979663904"] Jul 22 17:27:13.301836 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.301817 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a7d2796-0f06-47f2-ba76-f6dc74313f33" containerName="test-curl-7542-140412979739824" Jul 22 17:27:13.301916 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.301838 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a7d2796-0f06-47f2-ba76-f6dc74313f33" containerName="test-curl-7542-140412979739824" Jul 22 17:27:13.301971 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.301941 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a7d2796-0f06-47f2-ba76-f6dc74313f33" containerName="test-curl-7542-140412979739824" Jul 22 17:27:13.304875 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.304855 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:13.312090 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.312064 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979663904"] Jul 22 17:27:13.325712 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.325690 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttv94\" (UniqueName: \"kubernetes.io/projected/8a7d2796-0f06-47f2-ba76-f6dc74313f33-kube-api-access-ttv94\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:13.426283 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.426256 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx5mk\" (UniqueName: \"kubernetes.io/projected/4635ca0b-49e7-4bc3-8036-183af300cda3-kube-api-access-tx5mk\") pod \"test-curl-7542-140412979663904\" (UID: \"4635ca0b-49e7-4bc3-8036-183af300cda3\") " pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:13.527490 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.527426 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx5mk\" (UniqueName: \"kubernetes.io/projected/4635ca0b-49e7-4bc3-8036-183af300cda3-kube-api-access-tx5mk\") pod \"test-curl-7542-140412979663904\" (UID: \"4635ca0b-49e7-4bc3-8036-183af300cda3\") " pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:13.536130 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.536108 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx5mk\" (UniqueName: \"kubernetes.io/projected/4635ca0b-49e7-4bc3-8036-183af300cda3-kube-api-access-tx5mk\") pod \"test-curl-7542-140412979663904\" (UID: \"4635ca0b-49e7-4bc3-8036-183af300cda3\") " pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:13.615118 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.615096 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:13.622698 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.622668 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a7d2796-0f06-47f2-ba76-f6dc74313f33" path="/var/lib/kubelet/pods/8a7d2796-0f06-47f2-ba76-f6dc74313f33/volumes" Jul 22 17:27:13.746483 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.746459 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979663904"] Jul 22 17:27:13.748066 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:27:13.748037 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4635ca0b_49e7_4bc3_8036_183af300cda3.slice/crio-cfd188f87d3194d01b60395f64a6a81d5c77bd91fa3dcb6f50fc433090f193cd WatchSource:0}: Error finding container cfd188f87d3194d01b60395f64a6a81d5c77bd91fa3dcb6f50fc433090f193cd: Status 404 returned error can't find the container with id cfd188f87d3194d01b60395f64a6a81d5c77bd91fa3dcb6f50fc433090f193cd Jul 22 17:27:13.975232 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.975200 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979663904" event={"ID":"4635ca0b-49e7-4bc3-8036-183af300cda3","Type":"ContainerStarted","Data":"cfd188f87d3194d01b60395f64a6a81d5c77bd91fa3dcb6f50fc433090f193cd"} Jul 22 17:27:13.976474 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.976456 2581 scope.go:117] "RemoveContainer" containerID="fc48019f5047e17e6277e60295e619a3cf32fb2b762e43e427b1638b37452e9c" Jul 22 17:27:13.976597 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:13.976456 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979739824" Jul 22 17:27:14.982994 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:14.982958 2581 generic.go:356] "Generic (PLEG): container finished" podID="4635ca0b-49e7-4bc3-8036-183af300cda3" containerID="c78e7756d81c9ff136913dfe4040f1df76b7de53e295a1cacc6b9e4b84991acd" exitCode=0 Jul 22 17:27:14.983397 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:14.983044 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979663904" event={"ID":"4635ca0b-49e7-4bc3-8036-183af300cda3","Type":"ContainerDied","Data":"c78e7756d81c9ff136913dfe4040f1df76b7de53e295a1cacc6b9e4b84991acd"} Jul 22 17:27:16.117861 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.117838 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:16.137485 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.137459 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7542-140412979663904_4635ca0b-49e7-4bc3-8036-183af300cda3/test-curl-7542-140412979663904/0.log" Jul 22 17:27:16.159562 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.159536 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7542-140412979663904"] Jul 22 17:27:16.163502 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.163481 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7542-140412979663904"] Jul 22 17:27:16.250202 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.250138 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx5mk\" (UniqueName: \"kubernetes.io/projected/4635ca0b-49e7-4bc3-8036-183af300cda3-kube-api-access-tx5mk\") pod \"4635ca0b-49e7-4bc3-8036-183af300cda3\" (UID: \"4635ca0b-49e7-4bc3-8036-183af300cda3\") " Jul 22 17:27:16.252151 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.252125 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4635ca0b-49e7-4bc3-8036-183af300cda3-kube-api-access-tx5mk" (OuterVolumeSpecName: "kube-api-access-tx5mk") pod "4635ca0b-49e7-4bc3-8036-183af300cda3" (UID: "4635ca0b-49e7-4bc3-8036-183af300cda3"). InnerVolumeSpecName "kube-api-access-tx5mk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:27:16.290773 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.290746 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7542-140412979663904"] Jul 22 17:27:16.291108 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.291095 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4635ca0b-49e7-4bc3-8036-183af300cda3" containerName="test-curl-7542-140412979663904" Jul 22 17:27:16.291153 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.291109 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4635ca0b-49e7-4bc3-8036-183af300cda3" containerName="test-curl-7542-140412979663904" Jul 22 17:27:16.291186 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.291172 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4635ca0b-49e7-4bc3-8036-183af300cda3" containerName="test-curl-7542-140412979663904" Jul 22 17:27:16.294106 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.294090 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:16.300920 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.300898 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979663904"] Jul 22 17:27:16.351593 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.351565 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nrtb\" (UniqueName: \"kubernetes.io/projected/abdf6afa-6e82-409a-b7d8-c4c741d518e7-kube-api-access-5nrtb\") pod \"test-curl-7542-140412979663904\" (UID: \"abdf6afa-6e82-409a-b7d8-c4c741d518e7\") " pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:16.351719 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.351674 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx5mk\" (UniqueName: \"kubernetes.io/projected/4635ca0b-49e7-4bc3-8036-183af300cda3-kube-api-access-tx5mk\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:16.452496 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.452470 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nrtb\" (UniqueName: \"kubernetes.io/projected/abdf6afa-6e82-409a-b7d8-c4c741d518e7-kube-api-access-5nrtb\") pod \"test-curl-7542-140412979663904\" (UID: \"abdf6afa-6e82-409a-b7d8-c4c741d518e7\") " pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:16.461095 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.461075 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nrtb\" (UniqueName: \"kubernetes.io/projected/abdf6afa-6e82-409a-b7d8-c4c741d518e7-kube-api-access-5nrtb\") pod \"test-curl-7542-140412979663904\" (UID: \"abdf6afa-6e82-409a-b7d8-c4c741d518e7\") " pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:16.604589 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.604526 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:16.738854 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.738820 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979663904"] Jul 22 17:27:16.742063 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:27:16.742036 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabdf6afa_6e82_409a_b7d8_c4c741d518e7.slice/crio-eebcc088c2b1a9625e3b9ba7e53030f244c7acf2460d71c5c3b5adf55d736baa WatchSource:0}: Error finding container eebcc088c2b1a9625e3b9ba7e53030f244c7acf2460d71c5c3b5adf55d736baa: Status 404 returned error can't find the container with id eebcc088c2b1a9625e3b9ba7e53030f244c7acf2460d71c5c3b5adf55d736baa Jul 22 17:27:16.994915 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.994893 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:16.994915 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.994906 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfd188f87d3194d01b60395f64a6a81d5c77bd91fa3dcb6f50fc433090f193cd" Jul 22 17:27:16.996025 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:16.996004 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979663904" event={"ID":"abdf6afa-6e82-409a-b7d8-c4c741d518e7","Type":"ContainerStarted","Data":"eebcc088c2b1a9625e3b9ba7e53030f244c7acf2460d71c5c3b5adf55d736baa"} Jul 22 17:27:17.007846 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:17.007819 2581 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7542-140412979663904" oldPodUID="4635ca0b-49e7-4bc3-8036-183af300cda3" podUID="abdf6afa-6e82-409a-b7d8-c4c741d518e7" Jul 22 17:27:17.618702 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:17.618665 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4635ca0b-49e7-4bc3-8036-183af300cda3" path="/var/lib/kubelet/pods/4635ca0b-49e7-4bc3-8036-183af300cda3/volumes" Jul 22 17:27:18.002521 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:18.002491 2581 generic.go:356] "Generic (PLEG): container finished" podID="abdf6afa-6e82-409a-b7d8-c4c741d518e7" containerID="eadb262b46633b46997d22fbca57b292f33e5414960ed9fbe7a5881bc5d39d50" exitCode=0 Jul 22 17:27:18.002679 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:18.002540 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979663904" event={"ID":"abdf6afa-6e82-409a-b7d8-c4c741d518e7","Type":"ContainerDied","Data":"eadb262b46633b46997d22fbca57b292f33e5414960ed9fbe7a5881bc5d39d50"} Jul 22 17:27:19.146928 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.146901 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:19.167290 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.167264 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7542-140412979663904_abdf6afa-6e82-409a-b7d8-c4c741d518e7/test-curl-7542-140412979663904/0.log" Jul 22 17:27:19.190656 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.190619 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7542-140412979663904"] Jul 22 17:27:19.192913 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.192891 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7542-140412979663904"] Jul 22 17:27:19.275872 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.275823 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nrtb\" (UniqueName: \"kubernetes.io/projected/abdf6afa-6e82-409a-b7d8-c4c741d518e7-kube-api-access-5nrtb\") pod \"abdf6afa-6e82-409a-b7d8-c4c741d518e7\" (UID: \"abdf6afa-6e82-409a-b7d8-c4c741d518e7\") " Jul 22 17:27:19.277709 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.277684 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abdf6afa-6e82-409a-b7d8-c4c741d518e7-kube-api-access-5nrtb" (OuterVolumeSpecName: "kube-api-access-5nrtb") pod "abdf6afa-6e82-409a-b7d8-c4c741d518e7" (UID: "abdf6afa-6e82-409a-b7d8-c4c741d518e7"). InnerVolumeSpecName "kube-api-access-5nrtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:27:19.315880 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.315860 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7542-140412979664912"] Jul 22 17:27:19.316199 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.316188 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abdf6afa-6e82-409a-b7d8-c4c741d518e7" containerName="test-curl-7542-140412979663904" Jul 22 17:27:19.316249 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.316201 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="abdf6afa-6e82-409a-b7d8-c4c741d518e7" containerName="test-curl-7542-140412979663904" Jul 22 17:27:19.316286 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.316258 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="abdf6afa-6e82-409a-b7d8-c4c741d518e7" containerName="test-curl-7542-140412979663904" Jul 22 17:27:19.319054 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.319038 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:19.323826 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.323808 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979664912"] Jul 22 17:27:19.376826 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.376801 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rtnr\" (UniqueName: \"kubernetes.io/projected/3af8c0a6-4b74-464e-ae45-2c04b6f8c133-kube-api-access-4rtnr\") pod \"test-curl-7542-140412979664912\" (UID: \"3af8c0a6-4b74-464e-ae45-2c04b6f8c133\") " pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:19.376916 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.376907 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nrtb\" (UniqueName: \"kubernetes.io/projected/abdf6afa-6e82-409a-b7d8-c4c741d518e7-kube-api-access-5nrtb\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:19.477686 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.477664 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rtnr\" (UniqueName: \"kubernetes.io/projected/3af8c0a6-4b74-464e-ae45-2c04b6f8c133-kube-api-access-4rtnr\") pod \"test-curl-7542-140412979664912\" (UID: \"3af8c0a6-4b74-464e-ae45-2c04b6f8c133\") " pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:19.486537 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.486512 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rtnr\" (UniqueName: \"kubernetes.io/projected/3af8c0a6-4b74-464e-ae45-2c04b6f8c133-kube-api-access-4rtnr\") pod \"test-curl-7542-140412979664912\" (UID: \"3af8c0a6-4b74-464e-ae45-2c04b6f8c133\") " pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:19.618387 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.618320 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abdf6afa-6e82-409a-b7d8-c4c741d518e7" path="/var/lib/kubelet/pods/abdf6afa-6e82-409a-b7d8-c4c741d518e7/volumes" Jul 22 17:27:19.629689 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.629674 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:19.763429 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:19.763357 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979664912"] Jul 22 17:27:19.764090 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:27:19.764059 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3af8c0a6_4b74_464e_ae45_2c04b6f8c133.slice/crio-2ff03ef23e03d248b7d05f690632fd0504c8596ffcfe64fec2319e001b91e2cf WatchSource:0}: Error finding container 2ff03ef23e03d248b7d05f690632fd0504c8596ffcfe64fec2319e001b91e2cf: Status 404 returned error can't find the container with id 2ff03ef23e03d248b7d05f690632fd0504c8596ffcfe64fec2319e001b91e2cf Jul 22 17:27:20.014015 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:20.013984 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979664912" event={"ID":"3af8c0a6-4b74-464e-ae45-2c04b6f8c133","Type":"ContainerStarted","Data":"2ff03ef23e03d248b7d05f690632fd0504c8596ffcfe64fec2319e001b91e2cf"} Jul 22 17:27:20.015447 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:20.015424 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979663904" Jul 22 17:27:20.015551 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:20.015433 2581 scope.go:117] "RemoveContainer" containerID="eadb262b46633b46997d22fbca57b292f33e5414960ed9fbe7a5881bc5d39d50" Jul 22 17:27:21.023121 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.023081 2581 generic.go:356] "Generic (PLEG): container finished" podID="3af8c0a6-4b74-464e-ae45-2c04b6f8c133" containerID="e3f0276b7beef160ce916e1594bc052c0758efa9557f4a0e1643c1a5ae33bf96" exitCode=0 Jul 22 17:27:21.023510 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.023173 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979664912" event={"ID":"3af8c0a6-4b74-464e-ae45-2c04b6f8c133","Type":"ContainerDied","Data":"e3f0276b7beef160ce916e1594bc052c0758efa9557f4a0e1643c1a5ae33bf96"} Jul 22 17:27:21.239856 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.239819 2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" podUID="a80bca02-f840-4798-8e23-88ab29dfa43f" containerName="main" containerID="cri-o://20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520" gracePeriod=45 Jul 22 17:27:21.496559 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.496539 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:27:21.596288 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596260 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a80bca02-f840-4798-8e23-88ab29dfa43f-tls-certs\") pod \"a80bca02-f840-4798-8e23-88ab29dfa43f\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " Jul 22 17:27:21.596432 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596297 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-tmp-dir\") pod \"a80bca02-f840-4798-8e23-88ab29dfa43f\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " Jul 22 17:27:21.596432 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596357 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-model-cache\") pod \"a80bca02-f840-4798-8e23-88ab29dfa43f\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " Jul 22 17:27:21.596432 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596411 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-dshm\") pod \"a80bca02-f840-4798-8e23-88ab29dfa43f\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " Jul 22 17:27:21.596606 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596441 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xz6c2\" (UniqueName: \"kubernetes.io/projected/a80bca02-f840-4798-8e23-88ab29dfa43f-kube-api-access-xz6c2\") pod \"a80bca02-f840-4798-8e23-88ab29dfa43f\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " Jul 22 17:27:21.596606 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596464 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-home\") pod \"a80bca02-f840-4798-8e23-88ab29dfa43f\" (UID: \"a80bca02-f840-4798-8e23-88ab29dfa43f\") " Jul 22 17:27:21.596707 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596633 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a80bca02-f840-4798-8e23-88ab29dfa43f" (UID: "a80bca02-f840-4798-8e23-88ab29dfa43f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:27:21.596707 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596657 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-model-cache" (OuterVolumeSpecName: "model-cache") pod "a80bca02-f840-4798-8e23-88ab29dfa43f" (UID: "a80bca02-f840-4798-8e23-88ab29dfa43f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:27:21.596813 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596743 2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-tmp-dir\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:21.596813 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596791 2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-model-cache\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:21.596927 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.596809 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-home" (OuterVolumeSpecName: "home") pod "a80bca02-f840-4798-8e23-88ab29dfa43f" (UID: "a80bca02-f840-4798-8e23-88ab29dfa43f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:27:21.598337 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.598308 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a80bca02-f840-4798-8e23-88ab29dfa43f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a80bca02-f840-4798-8e23-88ab29dfa43f" (UID: "a80bca02-f840-4798-8e23-88ab29dfa43f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 17:27:21.598465 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.598416 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-dshm" (OuterVolumeSpecName: "dshm") pod "a80bca02-f840-4798-8e23-88ab29dfa43f" (UID: "a80bca02-f840-4798-8e23-88ab29dfa43f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 17:27:21.598517 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.598458 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a80bca02-f840-4798-8e23-88ab29dfa43f-kube-api-access-xz6c2" (OuterVolumeSpecName: "kube-api-access-xz6c2") pod "a80bca02-f840-4798-8e23-88ab29dfa43f" (UID: "a80bca02-f840-4798-8e23-88ab29dfa43f"). InnerVolumeSpecName "kube-api-access-xz6c2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:27:21.697742 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.697720 2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-dshm\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:21.697742 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.697740 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xz6c2\" (UniqueName: \"kubernetes.io/projected/a80bca02-f840-4798-8e23-88ab29dfa43f-kube-api-access-xz6c2\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:21.697874 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.697749 2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a80bca02-f840-4798-8e23-88ab29dfa43f-home\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:21.697874 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:21.697758 2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a80bca02-f840-4798-8e23-88ab29dfa43f-tls-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:22.029938 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.029906 2581 generic.go:356] "Generic (PLEG): container finished" podID="a80bca02-f840-4798-8e23-88ab29dfa43f" containerID="20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520" exitCode=0 Jul 22 17:27:22.030327 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.029945 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" Jul 22 17:27:22.030327 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.029990 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" event={"ID":"a80bca02-f840-4798-8e23-88ab29dfa43f","Type":"ContainerDied","Data":"20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520"} Jul 22 17:27:22.030327 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.030031 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2" event={"ID":"a80bca02-f840-4798-8e23-88ab29dfa43f","Type":"ContainerDied","Data":"93a545ae7c7a9edfa955df741f9a5439aa8df863e12212e777a51a18b2343e1d"} Jul 22 17:27:22.030327 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.030048 2581 scope.go:117] "RemoveContainer" containerID="20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520" Jul 22 17:27:22.039115 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.039098 2581 scope.go:117] "RemoveContainer" containerID="20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520" Jul 22 17:27:22.039424 ip-10-0-132-57 kubenswrapper[2581]: E0722 17:27:22.039405 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520\": container with ID starting with 20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520 not found: ID does not exist" containerID="20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520" Jul 22 17:27:22.039482 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.039434 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520"} err="failed to get container status \"20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520\": rpc error: code = NotFound desc = could not find container \"20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520\": container with ID starting with 20eea408d15bfcfee6265f7c0c2bb47ba2aae3bf85f92c4252efe1aeef4f8520 not found: ID does not exist" Jul 22 17:27:22.048899 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.048875 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2"] Jul 22 17:27:22.053600 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.053580 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-05e61d/rate-test-model-05e61d-kserve-5876bbf5f-zcnz2"] Jul 22 17:27:22.152427 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.152405 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:22.172154 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.172132 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7542-140412979664912_3af8c0a6-4b74-464e-ae45-2c04b6f8c133/test-curl-7542-140412979664912/0.log" Jul 22 17:27:22.195905 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.195878 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7542-140412979664912"] Jul 22 17:27:22.199426 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.199402 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7542-140412979664912"] Jul 22 17:27:22.303209 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.303154 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rtnr\" (UniqueName: \"kubernetes.io/projected/3af8c0a6-4b74-464e-ae45-2c04b6f8c133-kube-api-access-4rtnr\") pod \"3af8c0a6-4b74-464e-ae45-2c04b6f8c133\" (UID: \"3af8c0a6-4b74-464e-ae45-2c04b6f8c133\") " Jul 22 17:27:22.305129 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.305106 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af8c0a6-4b74-464e-ae45-2c04b6f8c133-kube-api-access-4rtnr" (OuterVolumeSpecName: "kube-api-access-4rtnr") pod "3af8c0a6-4b74-464e-ae45-2c04b6f8c133" (UID: "3af8c0a6-4b74-464e-ae45-2c04b6f8c133"). InnerVolumeSpecName "kube-api-access-4rtnr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:27:22.318696 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.318668 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7542-140412979664912"] Jul 22 17:27:22.319000 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.318988 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a80bca02-f840-4798-8e23-88ab29dfa43f" containerName="main" Jul 22 17:27:22.319045 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.319003 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a80bca02-f840-4798-8e23-88ab29dfa43f" containerName="main" Jul 22 17:27:22.319045 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.319023 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3af8c0a6-4b74-464e-ae45-2c04b6f8c133" containerName="test-curl-7542-140412979664912" Jul 22 17:27:22.319045 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.319029 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3af8c0a6-4b74-464e-ae45-2c04b6f8c133" containerName="test-curl-7542-140412979664912" Jul 22 17:27:22.319134 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.319083 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a80bca02-f840-4798-8e23-88ab29dfa43f" containerName="main" Jul 22 17:27:22.319134 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.319092 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3af8c0a6-4b74-464e-ae45-2c04b6f8c133" containerName="test-curl-7542-140412979664912" Jul 22 17:27:22.321958 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.321944 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:22.329185 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.329163 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979664912"] Jul 22 17:27:22.404173 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.404151 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64m9z\" (UniqueName: \"kubernetes.io/projected/c104b70b-c011-4332-95eb-5bb6a98ee5d1-kube-api-access-64m9z\") pod \"test-curl-7542-140412979664912\" (UID: \"c104b70b-c011-4332-95eb-5bb6a98ee5d1\") " pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:22.404269 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.404227 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rtnr\" (UniqueName: \"kubernetes.io/projected/3af8c0a6-4b74-464e-ae45-2c04b6f8c133-kube-api-access-4rtnr\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:22.505152 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.505128 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64m9z\" (UniqueName: \"kubernetes.io/projected/c104b70b-c011-4332-95eb-5bb6a98ee5d1-kube-api-access-64m9z\") pod \"test-curl-7542-140412979664912\" (UID: \"c104b70b-c011-4332-95eb-5bb6a98ee5d1\") " pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:22.520665 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.520648 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64m9z\" (UniqueName: \"kubernetes.io/projected/c104b70b-c011-4332-95eb-5bb6a98ee5d1-kube-api-access-64m9z\") pod \"test-curl-7542-140412979664912\" (UID: \"c104b70b-c011-4332-95eb-5bb6a98ee5d1\") " pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:22.631816 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.631756 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:22.762232 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:22.762204 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7542-140412979664912"] Jul 22 17:27:22.763267 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:27:22.763245 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc104b70b_c011_4332_95eb_5bb6a98ee5d1.slice/crio-1509c1eb0b098d55b4e09e9e31a15b66d73b34039e7c76fa64568aa99048bef2 WatchSource:0}: Error finding container 1509c1eb0b098d55b4e09e9e31a15b66d73b34039e7c76fa64568aa99048bef2: Status 404 returned error can't find the container with id 1509c1eb0b098d55b4e09e9e31a15b66d73b34039e7c76fa64568aa99048bef2 Jul 22 17:27:23.036507 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:23.036481 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:23.036901 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:23.036514 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ff03ef23e03d248b7d05f690632fd0504c8596ffcfe64fec2319e001b91e2cf" Jul 22 17:27:23.038430 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:23.038402 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979664912" event={"ID":"c104b70b-c011-4332-95eb-5bb6a98ee5d1","Type":"ContainerStarted","Data":"1509c1eb0b098d55b4e09e9e31a15b66d73b34039e7c76fa64568aa99048bef2"} Jul 22 17:27:23.049046 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:23.049025 2581 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7542-140412979664912" oldPodUID="3af8c0a6-4b74-464e-ae45-2c04b6f8c133" podUID="c104b70b-c011-4332-95eb-5bb6a98ee5d1" Jul 22 17:27:23.619530 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:23.619501 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3af8c0a6-4b74-464e-ae45-2c04b6f8c133" path="/var/lib/kubelet/pods/3af8c0a6-4b74-464e-ae45-2c04b6f8c133/volumes" Jul 22 17:27:23.619886 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:23.619873 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a80bca02-f840-4798-8e23-88ab29dfa43f" path="/var/lib/kubelet/pods/a80bca02-f840-4798-8e23-88ab29dfa43f/volumes" Jul 22 17:27:24.051110 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:24.051071 2581 generic.go:356] "Generic (PLEG): container finished" podID="c104b70b-c011-4332-95eb-5bb6a98ee5d1" containerID="3e5b3045f5f277a2be6b6374e831b96b3905a183326adc7254fb104bffedbd4a" exitCode=0 Jul 22 17:27:24.051605 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:24.051156 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7542-140412979664912" event={"ID":"c104b70b-c011-4332-95eb-5bb6a98ee5d1","Type":"ContainerDied","Data":"3e5b3045f5f277a2be6b6374e831b96b3905a183326adc7254fb104bffedbd4a"} Jul 22 17:27:25.196278 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:25.196256 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:25.217309 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:25.217289 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7542-140412979664912_c104b70b-c011-4332-95eb-5bb6a98ee5d1/test-curl-7542-140412979664912/0.log" Jul 22 17:27:25.242636 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:25.242610 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7542-140412979664912"] Jul 22 17:27:25.245347 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:25.245326 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7542-140412979664912"] Jul 22 17:27:25.326621 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:25.326567 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64m9z\" (UniqueName: \"kubernetes.io/projected/c104b70b-c011-4332-95eb-5bb6a98ee5d1-kube-api-access-64m9z\") pod \"c104b70b-c011-4332-95eb-5bb6a98ee5d1\" (UID: \"c104b70b-c011-4332-95eb-5bb6a98ee5d1\") " Jul 22 17:27:25.328537 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:25.328511 2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c104b70b-c011-4332-95eb-5bb6a98ee5d1-kube-api-access-64m9z" (OuterVolumeSpecName: "kube-api-access-64m9z") pod "c104b70b-c011-4332-95eb-5bb6a98ee5d1" (UID: "c104b70b-c011-4332-95eb-5bb6a98ee5d1"). InnerVolumeSpecName "kube-api-access-64m9z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 17:27:25.427537 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:25.427514 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64m9z\" (UniqueName: \"kubernetes.io/projected/c104b70b-c011-4332-95eb-5bb6a98ee5d1-kube-api-access-64m9z\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Jul 22 17:27:25.618762 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:25.618700 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c104b70b-c011-4332-95eb-5bb6a98ee5d1" path="/var/lib/kubelet/pods/c104b70b-c011-4332-95eb-5bb6a98ee5d1/volumes" Jul 22 17:27:25.663920 ip-10-0-132-57 kubenswrapper[2581]: E0722 17:27:25.663890 2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc104b70b_c011_4332_95eb_5bb6a98ee5d1.slice\": RecentStats: unable to find data in memory cache]" Jul 22 17:27:26.063064 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:26.063034 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7542-140412979664912" Jul 22 17:27:26.063238 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:27:26.063079 2581 scope.go:117] "RemoveContainer" containerID="3e5b3045f5f277a2be6b6374e831b96b3905a183326adc7254fb104bffedbd4a" Jul 22 17:28:10.405667 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:10.405582 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-689b965f47-4wx72_fc927414-66ea-4c2a-a716-4be30de05f17/authorino/0.log" Jul 22 17:28:16.490955 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:16.490925 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4xpnv_01c0b29f-fa6c-4777-96f6-fb7d39a01f23/server/0.log" Jul 22 17:28:16.623749 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:16.623716 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bnw5n_f0709664-86d3-4650-8693-d835271dc46d/manager/0.log" Jul 22 17:28:17.012918 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:17.012893 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55b7bf7664-vz8ch_ddf1371f-2f06-4957-99b9-b7103af01681/copy-manifests/0.log" Jul 22 17:28:17.020944 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:17.020916 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55b7bf7664-vz8ch_ddf1371f-2f06-4957-99b9-b7103af01681/manager/0.log" Jul 22 17:28:18.566358 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.566317 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj_2f0ba89f-d2fe-4e82-af1c-58d794054c9c/extract/0.log" Jul 22 17:28:18.582438 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.582414 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj_2f0ba89f-d2fe-4e82-af1c-58d794054c9c/util/0.log" Jul 22 17:28:18.589773 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.589754 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj_2f0ba89f-d2fe-4e82-af1c-58d794054c9c/pull/0.log" Jul 22 17:28:18.703571 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.703551 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td_d4b0b269-0edf-4d86-be61-d2a6c6b75701/util/0.log" Jul 22 17:28:18.710629 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.710609 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td_d4b0b269-0edf-4d86-be61-d2a6c6b75701/pull/0.log" Jul 22 17:28:18.717667 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.717643 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td_d4b0b269-0edf-4d86-be61-d2a6c6b75701/extract/0.log" Jul 22 17:28:18.834062 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.834008 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s_1cc39293-34d0-408d-910f-79c8f4a3295d/util/0.log" Jul 22 17:28:18.840162 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.840145 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s_1cc39293-34d0-408d-910f-79c8f4a3295d/pull/0.log" Jul 22 17:28:18.846427 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.846409 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s_1cc39293-34d0-408d-910f-79c8f4a3295d/extract/0.log" Jul 22 17:28:18.954701 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.954667 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf_8dacb29f-cf97-4dc0-bd83-7a619c3f6085/util/0.log" Jul 22 17:28:18.961111 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.961095 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf_8dacb29f-cf97-4dc0-bd83-7a619c3f6085/pull/0.log" Jul 22 17:28:18.967504 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:18.967487 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf_8dacb29f-cf97-4dc0-bd83-7a619c3f6085/extract/0.log" Jul 22 17:28:19.085083 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:19.085025 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-689b965f47-4wx72_fc927414-66ea-4c2a-a716-4be30de05f17/authorino/0.log" Jul 22 17:28:19.455562 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:19.455497 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wrrx8_6843d17c-b31d-4c13-b84b-bd97fe892592/kuadrant-console-plugin/0.log" Jul 22 17:28:20.704754 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:20.704725 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c467c957f-cmt89_e8fd3d84-4847-4c3c-b815-e99e522f8225/kube-auth-proxy/0.log" Jul 22 17:28:21.214476 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:21.214451 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55cc957bbb-8brdb_f88cc0d4-8c3d-4c35-9326-ce054e6cf331/router/0.log" Jul 22 17:28:21.682238 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:21.682211 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-m6988_10ff4d4b-de14-47f4-bb4c-cebc4da6f860/main/0.log" Jul 22 17:28:21.941992 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:21.941921 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsgfvp_49d96579-47cd-4937-900a-b229e3db8830/main/0.log" Jul 22 17:28:22.163172 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:22.163144 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-thzq6_e9f1c8e6-d565-4310-9dbd-e9ed034332df/main/0.log" Jul 22 17:28:30.272713 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:30.272682 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dmtxl_e0eb8cbf-7a75-4b2e-8bf4-9439e5b8975a/global-pull-secret-syncer/0.log" Jul 22 17:28:30.456824 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:30.456794 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t6d5r_94fbec68-3b05-4289-89d9-21b2a420a758/konnectivity-agent/0.log" Jul 22 17:28:30.478111 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:30.478088 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-57.ec2.internal_fb8eed94bb9bf5d97c7ef83964eedb6a/haproxy/0.log" Jul 22 17:28:33.787911 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:33.787835 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj_2f0ba89f-d2fe-4e82-af1c-58d794054c9c/extract/0.log" Jul 22 17:28:33.812027 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:33.812005 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj_2f0ba89f-d2fe-4e82-af1c-58d794054c9c/util/0.log" Jul 22 17:28:33.834035 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:33.834016 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759b6hrj_2f0ba89f-d2fe-4e82-af1c-58d794054c9c/pull/0.log" Jul 22 17:28:33.862849 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:33.862831 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td_d4b0b269-0edf-4d86-be61-d2a6c6b75701/extract/0.log" Jul 22 17:28:33.886102 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:33.886080 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td_d4b0b269-0edf-4d86-be61-d2a6c6b75701/util/0.log" Jul 22 17:28:33.906653 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:33.906633 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05x8td_d4b0b269-0edf-4d86-be61-d2a6c6b75701/pull/0.log" Jul 22 17:28:33.941906 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:33.941880 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s_1cc39293-34d0-408d-910f-79c8f4a3295d/extract/0.log" Jul 22 17:28:33.971474 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:33.971453 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s_1cc39293-34d0-408d-910f-79c8f4a3295d/util/0.log" Jul 22 17:28:33.990468 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:33.990446 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hzk7s_1cc39293-34d0-408d-910f-79c8f4a3295d/pull/0.log" Jul 22 17:28:34.018327 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:34.018311 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf_8dacb29f-cf97-4dc0-bd83-7a619c3f6085/extract/0.log" Jul 22 17:28:34.045637 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:34.045589 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf_8dacb29f-cf97-4dc0-bd83-7a619c3f6085/util/0.log" Jul 22 17:28:34.065546 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:34.065525 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kdchf_8dacb29f-cf97-4dc0-bd83-7a619c3f6085/pull/0.log" Jul 22 17:28:34.384946 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:34.384851 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-689b965f47-4wx72_fc927414-66ea-4c2a-a716-4be30de05f17/authorino/0.log" Jul 22 17:28:34.493199 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:34.493170 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wrrx8_6843d17c-b31d-4c13-b84b-bd97fe892592/kuadrant-console-plugin/0.log" Jul 22 17:28:36.839407 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:36.839359 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-h8th8_a3c3c642-652a-4b5f-b150-04e8b0c7b074/cluster-monitoring-operator/0.log" Jul 22 17:28:37.158031 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:37.157957 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ql5bm_64f3f420-7bb6-45e9-832e-3601a5ee6b16/node-exporter/0.log" Jul 22 17:28:37.187413 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:37.187389 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ql5bm_64f3f420-7bb6-45e9-832e-3601a5ee6b16/kube-rbac-proxy/0.log" Jul 22 17:28:37.214226 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:37.214210 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ql5bm_64f3f420-7bb6-45e9-832e-3601a5ee6b16/init-textfile/0.log" Jul 22 17:28:37.674928 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:37.674895 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-xlscb_e2da78f6-f615-4ded-b42f-773e4aef5229/prometheus-operator-admission-webhook/0.log" Jul 22 17:28:37.708748 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:37.708723 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f74fd479d-dn26w_c333945b-4dac-4941-868d-a43d34fdff39/telemeter-client/0.log" Jul 22 17:28:37.735012 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:37.734990 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f74fd479d-dn26w_c333945b-4dac-4941-868d-a43d34fdff39/reload/0.log" Jul 22 17:28:37.750257 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:37.750239 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f74fd479d-dn26w_c333945b-4dac-4941-868d-a43d34fdff39/kube-rbac-proxy/0.log" Jul 22 17:28:39.295259 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:39.295232 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-f2pjb_60ea9185-8846-4c18-916a-8587d9c0877a/networking-console-plugin/0.log" Jul 22 17:28:39.968706 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:39.968675 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/2.log" Jul 22 17:28:39.972989 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:39.972967 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-m5hgc_3bde7f11-4339-4d78-8531-b12f8d734aec/console-operator/3.log" Jul 22 17:28:40.653499 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.653464 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4"] Jul 22 17:28:40.654030 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.654011 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c104b70b-c011-4332-95eb-5bb6a98ee5d1" containerName="test-curl-7542-140412979664912" Jul 22 17:28:40.654076 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.654034 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c104b70b-c011-4332-95eb-5bb6a98ee5d1" containerName="test-curl-7542-140412979664912" Jul 22 17:28:40.654152 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.654140 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c104b70b-c011-4332-95eb-5bb6a98ee5d1" containerName="test-curl-7542-140412979664912" Jul 22 17:28:40.656585 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.656566 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6fc48fb58d-fp4xd_889fad7f-aa13-4bef-9200-1e3a9f37f685/console/0.log" Jul 22 17:28:40.657800 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.657781 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.661933 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.661910 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-52llv\"/\"openshift-service-ca.crt\"" Jul 22 17:28:40.662316 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.661937 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-52llv\"/\"default-dockercfg-rh2ng\"" Jul 22 17:28:40.662760 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.661997 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-52llv\"/\"kube-root-ca.crt\"" Jul 22 17:28:40.667599 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.667577 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4"] Jul 22 17:28:40.815881 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.815859 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-podres\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.816002 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.815889 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-proc\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.816002 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.815914 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-sys\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.816078 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.816004 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jz5m\" (UniqueName: \"kubernetes.io/projected/076275b6-f41f-4010-8e71-d70c7cc0319a-kube-api-access-8jz5m\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.816078 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.816051 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-lib-modules\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.916537 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.916462 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-lib-modules\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.916537 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.916510 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-podres\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.916537 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.916530 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-proc\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.916750 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.916553 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-sys\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.916750 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.916588 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jz5m\" (UniqueName: \"kubernetes.io/projected/076275b6-f41f-4010-8e71-d70c7cc0319a-kube-api-access-8jz5m\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.916750 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.916634 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-podres\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.916750 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.916657 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-proc\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.916750 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.916642 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-lib-modules\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.916750 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.916684 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/076275b6-f41f-4010-8e71-d70c7cc0319a-sys\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.924597 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.924578 2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jz5m\" (UniqueName: \"kubernetes.io/projected/076275b6-f41f-4010-8e71-d70c7cc0319a-kube-api-access-8jz5m\") pod \"perf-node-gather-daemonset-b2rr4\" (UID: \"076275b6-f41f-4010-8e71-d70c7cc0319a\") " pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:40.973162 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:40.973138 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:41.101818 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:41.101691 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4"] Jul 22 17:28:41.103593 ip-10-0-132-57 kubenswrapper[2581]: W0722 17:28:41.103561 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod076275b6_f41f_4010_8e71_d70c7cc0319a.slice/crio-96aa75f08fcbf259b2b2aaa42de230acac914f0cc04dc00a5a33a654f203107c WatchSource:0}: Error finding container 96aa75f08fcbf259b2b2aaa42de230acac914f0cc04dc00a5a33a654f203107c: Status 404 returned error can't find the container with id 96aa75f08fcbf259b2b2aaa42de230acac914f0cc04dc00a5a33a654f203107c Jul 22 17:28:41.407836 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:41.407818 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-7l82q_170f3511-cd3a-4829-87e2-ffb803cd0b69/volume-data-source-validator/0.log" Jul 22 17:28:41.458736 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:41.458708 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" event={"ID":"076275b6-f41f-4010-8e71-d70c7cc0319a","Type":"ContainerStarted","Data":"3c171487de6de476b2a2dca368f0e84217840d1fc4f2ed68b614db66a462a023"} Jul 22 17:28:41.458850 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:41.458763 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" event={"ID":"076275b6-f41f-4010-8e71-d70c7cc0319a","Type":"ContainerStarted","Data":"96aa75f08fcbf259b2b2aaa42de230acac914f0cc04dc00a5a33a654f203107c"} Jul 22 17:28:41.458850 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:41.458834 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:41.475773 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:41.475733 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" podStartSLOduration=1.475720202 podStartE2EDuration="1.475720202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 17:28:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 17:28:41.473230249 +0000 UTC m=+2908.383887420" watchObservedRunningTime="2026-07-22 17:28:41.475720202 +0000 UTC m=+2908.386377374" Jul 22 17:28:42.466160 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:42.466133 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hxzdx_2463af8f-0af9-4ff6-ba3b-d944175ebca1/dns/0.log" Jul 22 17:28:42.486158 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:42.486136 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hxzdx_2463af8f-0af9-4ff6-ba3b-d944175ebca1/kube-rbac-proxy/0.log" Jul 22 17:28:42.557172 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:42.557152 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7tdk9_d1de09de-782e-4b78-a68d-cbf1078cbc90/dns-node-resolver/0.log" Jul 22 17:28:43.208216 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:43.208190 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nxfwf_b77686d0-8cd0-4908-98fb-f34ff9a63c79/node-ca/0.log" Jul 22 17:28:44.242757 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:44.242733 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c467c957f-cmt89_e8fd3d84-4847-4c3c-b815-e99e522f8225/kube-auth-proxy/0.log" Jul 22 17:28:44.419872 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:44.419840 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55cc957bbb-8brdb_f88cc0d4-8c3d-4c35-9326-ce054e6cf331/router/0.log" Jul 22 17:28:45.091932 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:45.091906 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4h6hw_191c6bf4-8759-47b4-87ae-077336aa181e/serve-healthcheck-canary/0.log" Jul 22 17:28:45.590851 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:45.590821 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-lv47m_fa257d23-f7ea-498f-a500-b653baad12c8/insights-operator/0.log" Jul 22 17:28:45.591577 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:45.591557 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-lv47m_fa257d23-f7ea-498f-a500-b653baad12c8/insights-operator/1.log" Jul 22 17:28:45.748666 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:45.748637 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gfzrg_a1409262-765e-45c5-97b1-df28ad5043ed/kube-rbac-proxy/0.log" Jul 22 17:28:45.769048 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:45.769021 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gfzrg_a1409262-765e-45c5-97b1-df28ad5043ed/exporter/0.log" Jul 22 17:28:45.789558 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:45.789536 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gfzrg_a1409262-765e-45c5-97b1-df28ad5043ed/extractor/0.log" Jul 22 17:28:47.473720 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:47.473683 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-52llv/perf-node-gather-daemonset-b2rr4" Jul 22 17:28:48.362642 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:48.362612 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4xpnv_01c0b29f-fa6c-4777-96f6-fb7d39a01f23/server/0.log" Jul 22 17:28:48.432408 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:48.432355 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bnw5n_f0709664-86d3-4650-8693-d835271dc46d/manager/0.log" Jul 22 17:28:48.670717 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:48.670632 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55b7bf7664-vz8ch_ddf1371f-2f06-4957-99b9-b7103af01681/manager/0.log" Jul 22 17:28:48.689873 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:48.689852 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55b7bf7664-vz8ch_ddf1371f-2f06-4957-99b9-b7103af01681/copy-manifests/0.log" Jul 22 17:28:50.048588 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:50.048565 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57844c55d9-5gb2w_c12eb5d9-8d51-40b4-914c-3e95eacf2379/manager/0.log" Jul 22 17:28:50.837333 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:50.837303 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c467c957f-cmt89_e8fd3d84-4847-4c3c-b815-e99e522f8225/kube-auth-proxy/0.log" Jul 22 17:28:50.977959 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:50.977934 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55cc957bbb-8brdb_f88cc0d4-8c3d-4c35-9326-ce054e6cf331/router/0.log" Jul 22 17:28:52.264848 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:52.264819 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c467c957f-cmt89_e8fd3d84-4847-4c3c-b815-e99e522f8225/kube-auth-proxy/0.log" Jul 22 17:28:52.409237 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:52.409210 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-55cc957bbb-8brdb_f88cc0d4-8c3d-4c35-9326-ce054e6cf331/router/0.log" Jul 22 17:28:55.583386 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:55.583335 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-8mgm7_7967225e-9262-443f-b76a-e532304e06f9/kube-storage-version-migrator-operator/1.log" Jul 22 17:28:55.584085 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:55.584066 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-8mgm7_7967225e-9262-443f-b76a-e532304e06f9/kube-storage-version-migrator-operator/0.log" Jul 22 17:28:56.529229 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:56.529197 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8xzv4_e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f/kube-multus-additional-cni-plugins/0.log" Jul 22 17:28:56.550606 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:56.550577 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8xzv4_e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f/egress-router-binary-copy/0.log" Jul 22 17:28:56.571650 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:56.571630 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8xzv4_e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f/cni-plugins/0.log" Jul 22 17:28:56.593784 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:56.593761 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8xzv4_e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f/bond-cni-plugin/0.log" Jul 22 17:28:56.616819 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:56.616803 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8xzv4_e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f/routeoverride-cni/0.log" Jul 22 17:28:56.638908 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:56.638887 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8xzv4_e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f/whereabouts-cni-bincopy/0.log" Jul 22 17:28:56.658635 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:56.658615 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8xzv4_e5b3ef7b-fe5d-41c9-9e86-9f5287a5496f/whereabouts-cni/0.log" Jul 22 17:28:57.062854 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:57.062831 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qj8th_fab8b243-8bf0-41c8-b56c-d7f75510b714/kube-multus/0.log" Jul 22 17:28:57.120952 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:57.120926 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ld4x4_eaf250c9-d810-4a19-9844-b0e5fa5b8526/network-metrics-daemon/0.log" Jul 22 17:28:57.137692 ip-10-0-132-57 kubenswrapper[2581]: I0722 17:28:57.137669 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ld4x4_eaf250c9-d810-4a19-9844-b0e5fa5b8526/kube-rbac-proxy/0.log"