Sep 05 06:23:20.450316 ip-10-0-142-24 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 05 06:23:20.450328 ip-10-0-142-24 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 05 06:23:20.450335 ip-10-0-142-24 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 05 06:23:20.450512 ip-10-0-142-24 systemd[1]: Failed to start Kubernetes Kubelet. Sep 05 06:23:30.554322 ip-10-0-142-24 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 05 06:23:30.554336 ip-10-0-142-24 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 99325531da124de68a7ccf01e19dd14a -- Sep 05 06:25:03.205770 ip-10-0-142-24 systemd[1]: Starting Kubernetes Kubelet... Sep 05 06:25:03.617536 ip-10-0-142-24 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 05 06:25:03.617536 ip-10-0-142-24 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 05 06:25:03.617536 ip-10-0-142-24 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 05 06:25:03.617536 ip-10-0-142-24 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 05 06:25:03.617536 ip-10-0-142-24 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 05 06:25:03.618624 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.618543 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 05 06:25:03.625951 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625929 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 06:25:03.625951 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625946 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 06:25:03.625951 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625950 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 06:25:03.625951 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625957 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625960 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625963 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625966 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625969 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625972 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625975 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625978 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625980 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625983 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625986 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625990 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 06:25:03.626112 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.625996 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626348 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626352 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626355 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626358 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626361 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626364 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626367 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626369 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626372 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626374 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626377 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626380 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626385 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626388 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626390 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626393 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626396 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626398 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626401 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 06:25:03.626403 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626403 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626406 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626409 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626412 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626415 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626422 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626426 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626430 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626433 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626436 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626439 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626441 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626444 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626447 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626449 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626452 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626455 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626458 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626463 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 06:25:03.626903 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626466 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626468 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626471 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626486 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626489 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626492 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626495 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626498 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626500 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626503 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626507 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626513 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626516 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626519 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626522 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626525 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626528 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626531 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626534 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 06:25:03.627369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626537 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626541 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626543 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626546 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626554 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626558 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626561 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626564 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626567 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626569 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626572 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626575 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626578 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626580 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626583 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626586 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626591 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626593 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626596 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626599 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 06:25:03.627889 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626601 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626604 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626607 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626609 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626612 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.626615 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627709 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627718 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627722 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627725 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627728 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627734 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627737 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627740 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627743 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627746 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627749 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627752 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627755 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 06:25:03.628369 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627760 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627763 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627768 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627772 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627778 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627780 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627783 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627786 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627788 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627791 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627794 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627797 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627800 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627802 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627805 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627807 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627810 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627815 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627818 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 06:25:03.628862 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627821 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627824 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627826 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627829 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627832 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627834 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627837 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627839 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627843 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627846 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627851 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627854 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627856 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627859 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627862 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627864 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627867 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627869 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627873 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627876 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 06:25:03.629327 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627878 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627881 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627886 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627889 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627891 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627894 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627897 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627900 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627903 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627905 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627908 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627911 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627913 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627916 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627918 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627923 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627926 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627929 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627931 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627934 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 06:25:03.629850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627937 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627940 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627943 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627946 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627948 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627951 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627954 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627960 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627963 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627965 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627968 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627971 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627974 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627976 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627979 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627982 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627984 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627987 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627990 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627995 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 06:25:03.630345 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.627998 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628001 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628003 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628006 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628008 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628011 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628013 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628625 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628632 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628636 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628639 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628645 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628649 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628652 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628655 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628658 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628660 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628663 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628666 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628668 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 06:25:03.630856 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628671 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628674 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628677 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628683 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628685 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628688 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628691 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628693 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628696 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628698 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628701 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628704 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628707 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628709 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628712 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628714 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628719 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628722 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628725 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628727 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 06:25:03.631342 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628730 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628733 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628746 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628750 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628754 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628756 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628759 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628762 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628764 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628769 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628772 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628775 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628777 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628780 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628782 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628785 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628788 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628791 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628793 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628796 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 06:25:03.631934 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628799 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628804 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628806 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628809 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628812 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628814 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628817 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628819 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628823 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628826 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628829 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628832 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628834 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628840 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628842 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628845 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628848 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628851 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628854 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628857 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 06:25:03.632426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628859 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628862 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628865 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628868 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628872 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628876 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628883 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628886 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628889 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628892 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628895 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628898 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628901 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628903 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628907 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628910 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628912 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628918 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628920 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 06:25:03.632956 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628924 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628928 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628930 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628933 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628936 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628939 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.628941 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.629063 2583 flags.go:64] FLAG: --address="0.0.0.0" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630029 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630045 2583 flags.go:64] FLAG: --anonymous-auth="true" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630050 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630054 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630057 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630061 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630065 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630069 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630071 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630075 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630078 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630081 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630084 2583 flags.go:64] FLAG: --cgroup-root="" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630087 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630090 2583 flags.go:64] FLAG: --client-ca-file="" Sep 05 06:25:03.633412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630093 2583 flags.go:64] FLAG: --cloud-provider="external" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630095 2583 flags.go:64] FLAG: --cluster-dns="[]" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630101 2583 flags.go:64] FLAG: --cluster-domain="" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630104 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630107 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630111 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630114 2583 flags.go:64] FLAG: --container-log-max-files="5" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630118 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630121 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630125 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630129 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630132 2583 flags.go:64] FLAG: --contention-profiling="false" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630137 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630139 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630145 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630147 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630153 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630156 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630161 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630165 2583 flags.go:64] FLAG: --enable-load-reader="false" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630169 2583 flags.go:64] FLAG: --enable-server="true" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630173 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630179 2583 flags.go:64] FLAG: --event-burst="100" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630182 2583 flags.go:64] FLAG: --event-qps="50" Sep 05 06:25:03.634017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630186 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630190 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630194 2583 flags.go:64] FLAG: --eviction-hard="" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630200 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630202 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630205 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630210 2583 flags.go:64] FLAG: --eviction-soft="" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630214 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630217 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630221 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630224 2583 flags.go:64] FLAG: --experimental-mounter-path="" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630229 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630232 2583 flags.go:64] FLAG: --fail-swap-on="true" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630235 2583 flags.go:64] FLAG: --feature-gates="" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630239 2583 flags.go:64] FLAG: --file-check-frequency="20s" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630242 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630245 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630248 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630251 2583 flags.go:64] FLAG: --healthz-port="10248" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630254 2583 flags.go:64] FLAG: --help="false" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630257 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-142-24.ec2.internal" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630259 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630263 2583 flags.go:64] FLAG: --http-check-frequency="20s" Sep 05 06:25:03.634617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630266 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630269 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630273 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630275 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630278 2583 flags.go:64] FLAG: --image-service-endpoint="" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630281 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630300 2583 flags.go:64] FLAG: --kube-api-burst="100" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630305 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630308 2583 flags.go:64] FLAG: --kube-api-qps="50" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630311 2583 flags.go:64] FLAG: --kube-reserved="" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630314 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630317 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630320 2583 flags.go:64] FLAG: --kubelet-cgroups="" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630323 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630326 2583 flags.go:64] FLAG: --lock-file="" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630329 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630332 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630335 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630340 2583 flags.go:64] FLAG: --log-json-split-stream="false" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630345 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630348 2583 flags.go:64] FLAG: --log-text-split-stream="false" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630351 2583 flags.go:64] FLAG: --logging-format="text" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630354 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630358 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 05 06:25:03.635206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630360 2583 flags.go:64] FLAG: --manifest-url="" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630365 2583 flags.go:64] FLAG: --manifest-url-header="" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630370 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630373 2583 flags.go:64] FLAG: --max-open-files="1000000" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630377 2583 flags.go:64] FLAG: --max-pods="110" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630380 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630384 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630387 2583 flags.go:64] FLAG: --memory-manager-policy="None" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630390 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630393 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630396 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630399 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630407 2583 flags.go:64] FLAG: --node-status-max-images="50" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630410 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630413 2583 flags.go:64] FLAG: --oom-score-adj="-999" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630417 2583 flags.go:64] FLAG: --pod-cidr="" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630419 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630425 2583 flags.go:64] FLAG: --pod-manifest-path="" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630428 2583 flags.go:64] FLAG: --pod-max-pids="-1" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630431 2583 flags.go:64] FLAG: --pods-per-core="0" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630434 2583 flags.go:64] FLAG: --port="10250" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630437 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630439 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0125580f278a2ae4c" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630442 2583 flags.go:64] FLAG: --qos-reserved="" Sep 05 06:25:03.635804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630445 2583 flags.go:64] FLAG: --read-only-port="10255" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630448 2583 flags.go:64] FLAG: --register-node="true" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630451 2583 flags.go:64] FLAG: --register-with-taints="" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630454 2583 flags.go:64] FLAG: --registry-burst="10" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630458 2583 flags.go:64] FLAG: --registry-qps="5" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630460 2583 flags.go:64] FLAG: --reserved-cpus="" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630463 2583 flags.go:64] FLAG: --reserved-memory="" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630467 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630470 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630485 2583 flags.go:64] FLAG: --rotate-certificates="false" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630488 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630491 2583 flags.go:64] FLAG: --runonce="false" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630494 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630497 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630500 2583 flags.go:64] FLAG: --seccomp-default="false" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630504 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630506 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630510 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630512 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630515 2583 flags.go:64] FLAG: --storage-driver-password="root" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630518 2583 flags.go:64] FLAG: --storage-driver-secure="false" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630521 2583 flags.go:64] FLAG: --storage-driver-table="stats" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630524 2583 flags.go:64] FLAG: --storage-driver-user="root" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630527 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630530 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 05 06:25:03.636425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630533 2583 flags.go:64] FLAG: --system-cgroups="" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630536 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630542 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630545 2583 flags.go:64] FLAG: --tls-cert-file="" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630547 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630551 2583 flags.go:64] FLAG: --tls-min-version="" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630554 2583 flags.go:64] FLAG: --tls-private-key-file="" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630557 2583 flags.go:64] FLAG: --topology-manager-policy="none" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630560 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630563 2583 flags.go:64] FLAG: --topology-manager-scope="container" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630566 2583 flags.go:64] FLAG: --v="2" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630569 2583 flags.go:64] FLAG: --version="false" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630574 2583 flags.go:64] FLAG: --vmodule="" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630578 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.630582 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630693 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630699 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630703 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630706 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630709 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630712 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630714 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 06:25:03.637055 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630718 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630721 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630723 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630726 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630728 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630731 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630734 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630737 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630739 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630742 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630745 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630747 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630750 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630753 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630756 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630759 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630761 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630763 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630766 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 06:25:03.637621 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630768 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630777 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630780 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630783 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630786 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630788 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630791 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630795 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630798 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630800 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630803 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630805 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630808 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630812 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630815 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630817 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630820 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630823 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630825 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630828 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 06:25:03.638132 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630830 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630833 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630836 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630838 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630841 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630843 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630846 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630849 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630852 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630855 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630859 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630862 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630865 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630868 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630871 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630874 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630876 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630880 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630882 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630886 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 06:25:03.638653 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630888 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630891 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630893 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630897 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630901 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630905 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630908 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630910 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630913 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630916 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630918 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630921 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630923 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630926 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630928 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630931 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630934 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630936 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630939 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 06:25:03.639143 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630941 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630944 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630947 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630949 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630952 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630954 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630957 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630959 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630962 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630964 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630967 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630970 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630974 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.630977 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.631872 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.639585 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 05 06:25:03.639641 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.639600 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 05 06:25:03.640492 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.640472 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 05 06:25:03.640492 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.640491 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 05 06:25:03.640556 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640547 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 06:25:03.640556 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640554 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640558 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640561 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640565 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640569 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640572 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640574 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640577 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640580 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640583 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640585 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640588 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640591 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640593 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640596 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640598 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640601 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640604 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640606 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 06:25:03.640613 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640609 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640613 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640617 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640620 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640624 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640627 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640630 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640632 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640635 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640638 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640641 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640644 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640646 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640649 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640652 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640654 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640657 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640659 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640662 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640664 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 06:25:03.641080 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640667 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640669 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640672 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640674 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640677 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640679 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640682 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640684 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640687 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640690 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640695 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640699 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640703 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640709 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640715 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640719 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640722 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640724 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640728 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 06:25:03.641651 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640731 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640734 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640737 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640740 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640743 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640746 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640749 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640751 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640754 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640757 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640759 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640762 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640764 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640767 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640770 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640772 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640775 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640777 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640780 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640783 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 06:25:03.642119 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640785 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640788 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640791 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640793 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640796 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640798 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640801 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640804 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640807 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640810 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640812 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640815 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640818 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640820 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640823 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640825 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640828 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640831 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640833 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 06:25:03.642627 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640836 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.640841 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640947 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640952 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640955 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640958 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640961 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640963 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640967 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640969 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640972 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640974 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640977 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640981 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 05 06:25:03.643078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640984 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640987 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640990 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640993 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640996 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.640999 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641002 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641005 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641009 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641013 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641016 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641019 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641022 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641025 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641028 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641031 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641034 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641037 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641039 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 05 06:25:03.643426 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641041 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641044 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641047 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641049 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641052 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641054 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641057 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641060 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641062 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641065 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641067 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641070 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641072 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641075 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641078 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641080 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641083 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641086 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641089 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 05 06:25:03.643908 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641092 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641095 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641097 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641100 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641103 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641105 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641108 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641112 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641115 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641117 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641120 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641123 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641126 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641128 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641131 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641134 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641136 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641139 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641142 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641144 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 05 06:25:03.644383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641147 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641149 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641152 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641154 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641157 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641159 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641162 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641164 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641167 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641170 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641172 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641175 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641177 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641180 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641182 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641185 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641187 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641190 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641192 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641195 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 05 06:25:03.644897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641198 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641200 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641203 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641205 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641208 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641211 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641213 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641215 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:03.641218 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.641223 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.641313 2583 server.go:962] "Client rotation is on, will bootstrap in background" Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.643701 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.644769 2583 server.go:1019] "Starting client certificate rotation" Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.644869 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 05 06:25:03.645382 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.644905 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 05 06:25:03.648359 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.648345 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 05 06:25:03.651290 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.651265 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 05 06:25:03.662334 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.662319 2583 log.go:25] "Validated CRI v1 runtime API" Sep 05 06:25:03.666492 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.666459 2583 log.go:25] "Validated CRI v1 image API" Sep 05 06:25:03.667294 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.667281 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 05 06:25:03.671509 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.671470 2583 fs.go:135] Filesystem UUIDs: map[1ca262d1-8dd6-462e-9b2a-a055a7b6dd32:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 caeec033-483d-45ee-8a49-f1b5a344777e:/dev/nvme0n1p3] Sep 05 06:25:03.671583 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.671507 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 05 06:25:03.678224 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.678107 2583 manager.go:216] Machine: {Timestamp:2026-09-05 06:25:03.676669201 +0000 UTC m=+0.360174099 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100000 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bc06bcc6dda141de739ea76911641 SystemUUID:ec2bc06b-cc6d-da14-1de7-39ea76911641 BootID:99325531-da12-4de6-8a7c-cf01e19dd14a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:73:64:db:fa:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:73:64:db:fa:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:cc:05:36:16:d0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 05 06:25:03.678224 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.678215 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 05 06:25:03.678409 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.678359 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 05 06:25:03.678950 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.678933 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 05 06:25:03.682624 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.682596 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 05 06:25:03.682754 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.682626 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-24.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 05 06:25:03.684158 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.684148 2583 topology_manager.go:138] "Creating topology manager with none policy" Sep 05 06:25:03.684192 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.684160 2583 container_manager_linux.go:309] "Creating device plugin manager" Sep 05 06:25:03.684192 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.684168 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 05 06:25:03.685029 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.685019 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 05 06:25:03.686931 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.686921 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 05 06:25:03.691437 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.691426 2583 state_mem.go:36] "Initialized new in-memory state store" Sep 05 06:25:03.691564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.691554 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 05 06:25:03.691604 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.691585 2583 kubelet.go:486] "Attempting to sync node with API server" Sep 05 06:25:03.691604 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.691599 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 05 06:25:03.691676 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.691613 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 05 06:25:03.691676 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.691623 2583 kubelet.go:398] "Adding apiserver pod source" Sep 05 06:25:03.691676 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.691635 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 05 06:25:03.692021 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.692008 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 05 06:25:03.692054 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.692031 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 05 06:25:03.695784 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.695766 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 05 06:25:03.698820 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.698803 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 05 06:25:03.698893 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.698845 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 05 06:25:03.699039 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699028 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 05 06:25:03.699088 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699044 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 05 06:25:03.699088 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699050 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 05 06:25:03.699088 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699055 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 05 06:25:03.699088 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699062 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 05 06:25:03.699088 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699070 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 05 06:25:03.699759 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699746 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 05 06:25:03.699759 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699758 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 05 06:25:03.699842 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699765 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 05 06:25:03.699842 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699771 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 05 06:25:03.699842 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699787 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 05 06:25:03.699842 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.699795 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 05 06:25:03.700754 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.700731 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-24.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 05 06:25:03.700795 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.700732 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 05 06:25:03.701524 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.701514 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 05 06:25:03.701557 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.701525 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 05 06:25:03.703519 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.703501 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m68qb" Sep 05 06:25:03.705273 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.705262 2583 server.go:1268] "Started kubelet" Sep 05 06:25:03.705376 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.705351 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 05 06:25:03.705434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.705345 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 05 06:25:03.705434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.705410 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 05 06:25:03.706042 ip-10-0-142-24 systemd[1]: Started Kubernetes Kubelet. Sep 05 06:25:03.707215 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.707099 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 05 06:25:03.708919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.708266 2583 server.go:310] "Adding debug handlers to kubelet server" Sep 05 06:25:03.709982 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.709959 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 05 06:25:03.711407 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.711393 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m68qb" Sep 05 06:25:03.712886 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.712863 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 05 06:25:03.713793 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.713766 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 05 06:25:03.714068 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.714045 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:03.714545 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.714513 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 05 06:25:03.714545 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.714537 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 05 06:25:03.714682 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.714646 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-24.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 05 06:25:03.714966 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.714911 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 05 06:25:03.715052 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.714982 2583 reconstruct.go:97] "Volume reconstruction finished" Sep 05 06:25:03.715052 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.714990 2583 reconciler.go:29] "Reconciler: start to sync state" Sep 05 06:25:03.715207 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.715191 2583 factory.go:55] Registering systemd factory Sep 05 06:25:03.715257 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.715211 2583 factory.go:223] Registration of the systemd container factory successfully Sep 05 06:25:03.715576 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.714235 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-24.ec2.internal.18d258ad6157c6f9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-24.ec2.internal,UID:ip-10-0-142-24.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-24.ec2.internal,},FirstTimestamp:2026-09-05 06:25:03.705237241 +0000 UTC m=+0.388742140,LastTimestamp:2026-09-05 06:25:03.705237241 +0000 UTC m=+0.388742140,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-24.ec2.internal,}" Sep 05 06:25:03.715576 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.715512 2583 factory.go:153] Registering CRI-O factory Sep 05 06:25:03.715576 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.715525 2583 factory.go:223] Registration of the crio container factory successfully Sep 05 06:25:03.715776 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.715585 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 05 06:25:03.715776 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.715606 2583 factory.go:103] Registering Raw factory Sep 05 06:25:03.715776 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.715622 2583 manager.go:1196] Started watching for new ooms in manager Sep 05 06:25:03.716545 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.716529 2583 manager.go:349] Starting recovery of all containers Sep 05 06:25:03.718745 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.718716 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-24.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 05 06:25:03.718854 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.718741 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 05 06:25:03.727312 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.727294 2583 manager.go:354] Recovery completed Sep 05 06:25:03.731571 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.731559 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 06:25:03.733344 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.733326 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Sep 05 06:25:03.733404 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.733360 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 06:25:03.733404 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.733370 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Sep 05 06:25:03.733945 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.733928 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 05 06:25:03.733945 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.733942 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 05 06:25:03.734088 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.733957 2583 state_mem.go:36] "Initialized new in-memory state store" Sep 05 06:25:03.734088 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.733955 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 06:25:03.734819 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.734802 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Sep 05 06:25:03.734819 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.734827 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 06:25:03.734957 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.734841 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Sep 05 06:25:03.735946 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.735932 2583 policy_none.go:49] "None policy: Start" Sep 05 06:25:03.735946 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.735948 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 05 06:25:03.736033 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.735959 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 05 06:25:03.738000 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.737989 2583 policy_none.go:47] "Start" Sep 05 06:25:03.770206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.770186 2583 manager.go:337] "Starting Device Plugin manager" Sep 05 06:25:03.790389 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.770237 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 05 06:25:03.790389 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.770250 2583 server.go:87] "Starting device plugin registration server" Sep 05 06:25:03.790389 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.770519 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 05 06:25:03.790389 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.770533 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 05 06:25:03.790389 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.770741 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 05 06:25:03.790389 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.770846 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 05 06:25:03.790389 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.770854 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 05 06:25:03.790389 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.771355 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 05 06:25:03.790389 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.771409 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:03.814562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.814532 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 05 06:25:03.815804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.815787 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 05 06:25:03.815904 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.815844 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 05 06:25:03.815904 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.815873 2583 kubelet.go:2448] "Starting kubelet main sync loop" Sep 05 06:25:03.815998 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.815917 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 05 06:25:03.820105 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.820082 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 06:25:03.871146 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.871061 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 06:25:03.872094 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.872077 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Sep 05 06:25:03.872158 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.872112 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 06:25:03.872158 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.872127 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Sep 05 06:25:03.872158 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.872155 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-24.ec2.internal" Sep 05 06:25:03.879250 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.879235 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-24.ec2.internal" Sep 05 06:25:03.879304 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.879257 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-24.ec2.internal\": node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:03.891461 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.891440 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:03.916140 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.916111 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal"] Sep 05 06:25:03.916203 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.916177 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 06:25:03.917526 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.917509 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Sep 05 06:25:03.917632 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.917536 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 06:25:03.917632 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.917546 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Sep 05 06:25:03.919907 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.919894 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 06:25:03.920037 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.920023 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Sep 05 06:25:03.920078 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.920054 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 06:25:03.920610 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.920595 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Sep 05 06:25:03.920691 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.920619 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 06:25:03.920787 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.920633 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Sep 05 06:25:03.920898 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.920761 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Sep 05 06:25:03.920898 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.920896 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 06:25:03.921005 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.920911 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Sep 05 06:25:03.923310 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.923293 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Sep 05 06:25:03.923398 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.923324 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 05 06:25:03.924019 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.924002 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientMemory" Sep 05 06:25:03.924123 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.924027 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasNoDiskPressure" Sep 05 06:25:03.924123 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:03.924036 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeHasSufficientPID" Sep 05 06:25:03.947016 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.946991 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-24.ec2.internal\" not found" node="ip-10-0-142-24.ec2.internal" Sep 05 06:25:03.951198 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.951181 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-24.ec2.internal\" not found" node="ip-10-0-142-24.ec2.internal" Sep 05 06:25:03.991961 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:03.991941 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:04.016903 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.016884 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4f563eaece2c34e5022dfd42837612e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"4f563eaece2c34e5022dfd42837612e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.016959 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.016911 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f563eaece2c34e5022dfd42837612e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"4f563eaece2c34e5022dfd42837612e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.016959 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.016933 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/36b5895e28f6f5810ea58bd85d6e692d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" (UID: \"36b5895e28f6f5810ea58bd85d6e692d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.092611 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:04.092586 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:04.117917 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.117899 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4f563eaece2c34e5022dfd42837612e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"4f563eaece2c34e5022dfd42837612e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.117971 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.117962 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4f563eaece2c34e5022dfd42837612e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"4f563eaece2c34e5022dfd42837612e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.118013 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.117993 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f563eaece2c34e5022dfd42837612e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"4f563eaece2c34e5022dfd42837612e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.118048 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.118025 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/36b5895e28f6f5810ea58bd85d6e692d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" (UID: \"36b5895e28f6f5810ea58bd85d6e692d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.118083 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.118062 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/36b5895e28f6f5810ea58bd85d6e692d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-24.ec2.internal\" (UID: \"36b5895e28f6f5810ea58bd85d6e692d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.118115 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.118092 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f563eaece2c34e5022dfd42837612e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal\" (UID: \"4f563eaece2c34e5022dfd42837612e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.193221 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:04.193164 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:04.249639 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.249608 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.255399 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.255022 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Sep 05 06:25:04.293637 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:04.293487 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:04.394514 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:04.394489 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:04.495160 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:04.495090 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:04.595432 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:04.595390 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:04.644047 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.644016 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 05 06:25:04.644459 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.644159 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 05 06:25:04.695864 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:04.695832 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:04.718648 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.718613 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-04 06:20:03 +0000 UTC" deadline="2028-03-24 18:14:08.614615046 +0000 UTC" Sep 05 06:25:04.718648 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.718646 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13595h49m3.895972579s" Sep 05 06:25:04.728201 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.728182 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 06:25:04.797013 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:04.796948 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:04.897122 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:04.897064 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-24.ec2.internal\" not found" Sep 05 06:25:04.909966 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:04.909927 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b5895e28f6f5810ea58bd85d6e692d.slice/crio-edc767dde865ba7517aef4d6807a289e0c2bab47f11fb34fedb234902daa83d6 WatchSource:0}: Error finding container edc767dde865ba7517aef4d6807a289e0c2bab47f11fb34fedb234902daa83d6: Status 404 returned error can't find the container with id edc767dde865ba7517aef4d6807a289e0c2bab47f11fb34fedb234902daa83d6 Sep 05 06:25:04.910887 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:04.910868 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f563eaece2c34e5022dfd42837612e6.slice/crio-a8f975a22ae2ab94c80a3a0ca5139736f17ac2170b4d6508252c37e9cb5a25ce WatchSource:0}: Error finding container a8f975a22ae2ab94c80a3a0ca5139736f17ac2170b4d6508252c37e9cb5a25ce: Status 404 returned error can't find the container with id a8f975a22ae2ab94c80a3a0ca5139736f17ac2170b4d6508252c37e9cb5a25ce Sep 05 06:25:04.916210 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.916193 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 05 06:25:04.957017 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:04.956993 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 06:25:05.015562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.015507 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" Sep 05 06:25:05.016690 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.016676 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 06:25:05.026341 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.026323 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 05 06:25:05.027659 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.027647 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" Sep 05 06:25:05.042466 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.042449 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 05 06:25:05.693307 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.693277 2583 apiserver.go:52] "Watching apiserver" Sep 05 06:25:05.699998 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.699975 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 05 06:25:05.701362 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.701333 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-2t2vq","openshift-ovn-kubernetes/ovnkube-node-c8w2h","kube-system/global-pull-secret-syncer-vcd6d","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8","openshift-dns/node-resolver-m42bc","openshift-image-registry/node-ca-4bf69","openshift-multus/multus-7tt2w","openshift-network-diagnostics/network-check-target-dfdlw","kube-system/konnectivity-agent-kvx7p","kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal","openshift-cluster-node-tuning-operator/tuned-7bj9p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal","openshift-multus/multus-additional-cni-plugins-kglkm","openshift-multus/network-metrics-daemon-m8fn4"] Sep 05 06:25:05.704002 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.703981 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.706199 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.706176 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.707806 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.707081 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 05 06:25:05.707806 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.707138 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 05 06:25:05.707806 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.707171 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 05 06:25:05.707806 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.707356 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ct5dn\"" Sep 05 06:25:05.707806 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.707374 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 05 06:25:05.708240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.708216 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:05.708958 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.708665 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:05.711043 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.709888 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 05 06:25:05.711043 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.710512 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 05 06:25:05.711043 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.710668 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 05 06:25:05.711043 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.710738 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 05 06:25:05.711283 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.711078 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 05 06:25:05.712384 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.711507 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.712769 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.712720 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lk6g8\"" Sep 05 06:25:05.713148 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.712989 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 05 06:25:05.713731 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.713716 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 05 06:25:05.714076 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.714038 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:05.714744 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.714512 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 05 06:25:05.714744 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.714533 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 05 06:25:05.714744 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.714574 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 05 06:25:05.717024 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.716868 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wvcj9\"" Sep 05 06:25:05.717223 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.717185 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bn4jm\"" Sep 05 06:25:05.717314 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.717235 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 05 06:25:05.719031 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.719011 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 05 06:25:05.719111 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.719029 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 05 06:25:05.719571 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.719548 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:05.722047 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.721806 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 05 06:25:05.722047 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.721846 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 05 06:25:05.722047 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.721885 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 05 06:25:05.722248 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.722112 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.722248 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.722173 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-797lp\"" Sep 05 06:25:05.724502 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.724464 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:05.725154 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.724803 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 05 06:25:05.725154 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.724893 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 05 06:25:05.725154 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.724944 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w2jl5\"" Sep 05 06:25:05.726723 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.726692 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 05 06:25:05.726920 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.726902 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g79fb\"" Sep 05 06:25:05.726986 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.726928 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 05 06:25:05.727042 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.727022 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.727958 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.727928 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 05 06:25:05.728204 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728184 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-run-k8s-cni-cncf-io\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.728283 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728237 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-hostroot\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.728283 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728263 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-ovnkube-config\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.728385 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728301 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mmdg\" (UniqueName: \"kubernetes.io/projected/68080680-0551-487c-80ed-b93acb2f68f8-kube-api-access-8mmdg\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:05.728385 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728325 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/970378b4-f06a-43ba-857e-1e9f1ecc88cb-iptables-alerter-script\") pod \"iptables-alerter-2t2vq\" (UID: \"970378b4-f06a-43ba-857e-1e9f1ecc88cb\") " pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:05.728385 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728365 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f64eb380-73df-46a4-9766-dd8d99bc4625-host\") pod \"node-ca-4bf69\" (UID: \"f64eb380-73df-46a4-9766-dd8d99bc4625\") " pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:05.728553 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728385 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/277802fb-ac81-406a-83aa-2824daafcb00-cni-binary-copy\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.728553 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728418 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-run-netns\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.728553 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728444 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/277802fb-ac81-406a-83aa-2824daafcb00-multus-daemon-config\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.728553 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728469 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-systemd-units\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.728553 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728508 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-run-netns\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.728553 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728529 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-etc-openvswitch\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.728758 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728553 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hxd7\" (UniqueName: \"kubernetes.io/projected/970378b4-f06a-43ba-857e-1e9f1ecc88cb-kube-api-access-6hxd7\") pod \"iptables-alerter-2t2vq\" (UID: \"970378b4-f06a-43ba-857e-1e9f1ecc88cb\") " pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:05.728758 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728576 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hj4m\" (UniqueName: \"kubernetes.io/projected/f64eb380-73df-46a4-9766-dd8d99bc4625-kube-api-access-6hj4m\") pod \"node-ca-4bf69\" (UID: \"f64eb380-73df-46a4-9766-dd8d99bc4625\") " pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:05.728758 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728599 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2wjp\" (UniqueName: \"kubernetes.io/projected/277802fb-ac81-406a-83aa-2824daafcb00-kube-api-access-z2wjp\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.728758 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728637 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-slash\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.728758 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728663 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.728758 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728711 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-ovn-node-metrics-cert\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.728758 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728736 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-ovnkube-script-lib\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728797 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728856 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-var-lib-openvswitch\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728900 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-run-openvswitch\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728942 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-socket-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728966 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-registration-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.728994 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-device-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729018 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-system-cni-dir\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729053 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-var-lib-cni-bin\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729100 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-etc-kubernetes\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729123 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-cni-bin\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.729164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729148 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-etc-selinux\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729172 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-sys-fs\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729195 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-cnibin\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729218 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-multus-conf-dir\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729274 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-run-multus-certs\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729309 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-run-ovn\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729347 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-log-socket\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729464 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-cni-netd\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729551 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729578 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/970378b4-f06a-43ba-857e-1e9f1ecc88cb-host-slash\") pod \"iptables-alerter-2t2vq\" (UID: \"970378b4-f06a-43ba-857e-1e9f1ecc88cb\") " pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729620 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-var-lib-cni-multus\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729647 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 05 06:25:05.729675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729652 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-var-lib-kubelet\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729717 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-kubelet\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729721 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7l6lj\"" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729737 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729749 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-env-overrides\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729784 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks25k\" (UniqueName: \"kubernetes.io/projected/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-kube-api-access-ks25k\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729800 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729868 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v225\" (UniqueName: \"kubernetes.io/projected/5d24e878-c4a6-4670-898b-42d157dcf067-kube-api-access-8v225\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729905 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-run-systemd\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729932 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-node-log\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729961 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.729987 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f64eb380-73df-46a4-9766-dd8d99bc4625-serviceca\") pod \"node-ca-4bf69\" (UID: \"f64eb380-73df-46a4-9766-dd8d99bc4625\") " pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.730006 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-multus-cni-dir\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.730237 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.730025 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-multus-socket-dir-parent\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.731649 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.731623 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:05.731758 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.731698 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:05.731973 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.731954 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 05 06:25:05.732086 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.732070 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bqpbf\"" Sep 05 06:25:05.732184 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.732167 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 05 06:25:05.733836 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.733815 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:05.733928 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.733888 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:05.753681 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.753113 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tsbt4" Sep 05 06:25:05.760588 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.760564 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tsbt4" Sep 05 06:25:05.815920 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.815895 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 05 06:25:05.820803 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.820744 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" event={"ID":"4f563eaece2c34e5022dfd42837612e6","Type":"ContainerStarted","Data":"a8f975a22ae2ab94c80a3a0ca5139736f17ac2170b4d6508252c37e9cb5a25ce"} Sep 05 06:25:05.822061 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.822030 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" event={"ID":"36b5895e28f6f5810ea58bd85d6e692d","Type":"ContainerStarted","Data":"edc767dde865ba7517aef4d6807a289e0c2bab47f11fb34fedb234902daa83d6"} Sep 05 06:25:05.830560 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-multus-cni-dir\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.830674 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830576 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-run-k8s-cni-cncf-io\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.830674 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-hostroot\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.830674 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830620 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-ovnkube-config\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.830674 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830642 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mmdg\" (UniqueName: \"kubernetes.io/projected/68080680-0551-487c-80ed-b93acb2f68f8-kube-api-access-8mmdg\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:05.830674 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830652 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-run-k8s-cni-cncf-io\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.830674 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830666 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-sys\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830719 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w6td\" (UniqueName: \"kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td\") pod \"network-check-target-dfdlw\" (UID: \"be8cb322-a209-49e2-bb7b-22352fcf54c0\") " pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830733 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-hostroot\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830767 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/970378b4-f06a-43ba-857e-1e9f1ecc88cb-iptables-alerter-script\") pod \"iptables-alerter-2t2vq\" (UID: \"970378b4-f06a-43ba-857e-1e9f1ecc88cb\") " pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830792 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f64eb380-73df-46a4-9766-dd8d99bc4625-host\") pod \"node-ca-4bf69\" (UID: \"f64eb380-73df-46a4-9766-dd8d99bc4625\") " pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830799 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-multus-cni-dir\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830818 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/277802fb-ac81-406a-83aa-2824daafcb00-cni-binary-copy\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/277802fb-ac81-406a-83aa-2824daafcb00-multus-daemon-config\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830863 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-systemd-units\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830886 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-run-netns\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830889 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f64eb380-73df-46a4-9766-dd8d99bc4625-host\") pod \"node-ca-4bf69\" (UID: \"f64eb380-73df-46a4-9766-dd8d99bc4625\") " pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830908 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-etc-openvswitch\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.830965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830966 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-run-netns\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.830970 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-etc-openvswitch\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831008 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-ovnkube-script-lib\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831039 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hxd7\" (UniqueName: \"kubernetes.io/projected/970378b4-f06a-43ba-857e-1e9f1ecc88cb-kube-api-access-6hxd7\") pod \"iptables-alerter-2t2vq\" (UID: \"970378b4-f06a-43ba-857e-1e9f1ecc88cb\") " pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831065 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hj4m\" (UniqueName: \"kubernetes.io/projected/f64eb380-73df-46a4-9766-dd8d99bc4625-kube-api-access-6hj4m\") pod \"node-ca-4bf69\" (UID: \"f64eb380-73df-46a4-9766-dd8d99bc4625\") " pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831092 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-slash\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831146 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-socket-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831171 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-registration-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831199 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d8ad0f2-e276-402b-8270-c6194aa5270f-agent-certs\") pod \"konnectivity-agent-kvx7p\" (UID: \"5d8ad0f2-e276-402b-8270-c6194aa5270f\") " pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831225 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/93d036de-ea2d-406d-a61d-4f6c972acc71-dbus\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-device-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831279 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-lib-modules\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831302 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-host\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831328 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-etc-kubernetes\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831355 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-cni-bin\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831371 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-ovnkube-config\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.831581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831373 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/970378b4-f06a-43ba-857e-1e9f1ecc88cb-iptables-alerter-script\") pod \"iptables-alerter-2t2vq\" (UID: \"970378b4-f06a-43ba-857e-1e9f1ecc88cb\") " pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831380 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-sysconfig\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831442 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-registration-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831455 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-device-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831469 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/277802fb-ac81-406a-83aa-2824daafcb00-multus-daemon-config\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831533 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-systemd-units\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831535 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/277802fb-ac81-406a-83aa-2824daafcb00-cni-binary-copy\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831549 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-systemd\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831574 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-cni-bin\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831589 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-slash\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831587 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba4875c4-1642-481c-88b7-f59c243040e4-cnibin\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831692 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba4875c4-1642-481c-88b7-f59c243040e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831700 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-etc-kubernetes\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831727 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-multus-conf-dir\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831758 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-cni-netd\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831797 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-socket-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.831799 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831812 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-cni-netd\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.832300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831818 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-multus-conf-dir\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831829 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ba4875c4-1642-481c-88b7-f59c243040e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.831882 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs podName:68080680-0551-487c-80ed-b93acb2f68f8 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:06.331861675 +0000 UTC m=+3.015366561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs") pod "network-metrics-daemon-m8fn4" (UID: "68080680-0551-487c-80ed-b93acb2f68f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-var-lib-kubelet\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831952 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks25k\" (UniqueName: \"kubernetes.io/projected/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-kube-api-access-ks25k\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831964 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-ovnkube-script-lib\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.831979 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-sysctl-d\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832014 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-var-lib-kubelet\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832058 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba4875c4-1642-481c-88b7-f59c243040e4-system-cni-dir\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832103 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832167 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-kubernetes\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832196 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/302f2dc8-13ec-46b9-b694-f66048d4e2c3-hosts-file\") pod \"node-resolver-m42bc\" (UID: \"302f2dc8-13ec-46b9-b694-f66048d4e2c3\") " pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832220 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba4875c4-1642-481c-88b7-f59c243040e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832253 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f64eb380-73df-46a4-9766-dd8d99bc4625-serviceca\") pod \"node-ca-4bf69\" (UID: \"f64eb380-73df-46a4-9766-dd8d99bc4625\") " pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832259 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832279 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-multus-socket-dir-parent\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.833030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832356 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-modprobe-d\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832384 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-multus-socket-dir-parent\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832423 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkcv9\" (UniqueName: \"kubernetes.io/projected/302f2dc8-13ec-46b9-b694-f66048d4e2c3-kube-api-access-jkcv9\") pod \"node-resolver-m42bc\" (UID: \"302f2dc8-13ec-46b9-b694-f66048d4e2c3\") " pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832448 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-run-netns\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832501 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832529 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-run-netns\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832577 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-ovn-node-metrics-cert\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832596 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832630 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832649 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f64eb380-73df-46a4-9766-dd8d99bc4625-serviceca\") pod \"node-ca-4bf69\" (UID: \"f64eb380-73df-46a4-9766-dd8d99bc4625\") " pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832654 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-sysctl-conf\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832765 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-var-lib-kubelet\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832787 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcr8r\" (UniqueName: \"kubernetes.io/projected/85439f8e-7098-47a1-ae54-d6b83a534da8-kube-api-access-pcr8r\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832807 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2wjp\" (UniqueName: \"kubernetes.io/projected/277802fb-ac81-406a-83aa-2824daafcb00-kube-api-access-z2wjp\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832847 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-tuned\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832868 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsxn6\" (UniqueName: \"kubernetes.io/projected/ba4875c4-1642-481c-88b7-f59c243040e4-kube-api-access-rsxn6\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832952 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 05 06:25:05.833627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.832975 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-var-lib-openvswitch\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833012 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-run-openvswitch\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833040 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-etc-selinux\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833088 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-system-cni-dir\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833094 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-var-lib-openvswitch\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833117 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-var-lib-cni-bin\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833161 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-sys-fs\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833180 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-etc-selinux\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833189 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d8ad0f2-e276-402b-8270-c6194aa5270f-konnectivity-ca\") pod \"konnectivity-agent-kvx7p\" (UID: \"5d8ad0f2-e276-402b-8270-c6194aa5270f\") " pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833218 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-cnibin\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833233 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-run-openvswitch\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833273 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-run-multus-certs\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833278 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-var-lib-cni-bin\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833299 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-run-ovn\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833316 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-cnibin\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833328 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-log-socket\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833368 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-run-ovn\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.834348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833372 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833401 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/93d036de-ea2d-406d-a61d-4f6c972acc71-kubelet-config\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833424 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302f2dc8-13ec-46b9-b694-f66048d4e2c3-tmp-dir\") pod \"node-resolver-m42bc\" (UID: \"302f2dc8-13ec-46b9-b694-f66048d4e2c3\") " pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833445 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-system-cni-dir\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833454 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/970378b4-f06a-43ba-857e-1e9f1ecc88cb-host-slash\") pod \"iptables-alerter-2t2vq\" (UID: \"970378b4-f06a-43ba-857e-1e9f1ecc88cb\") " pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833517 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/970378b4-f06a-43ba-857e-1e9f1ecc88cb-host-slash\") pod \"iptables-alerter-2t2vq\" (UID: \"970378b4-f06a-43ba-857e-1e9f1ecc88cb\") " pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833517 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-run-multus-certs\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833546 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-log-socket\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833556 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833550 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-var-lib-cni-multus\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833593 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/277802fb-ac81-406a-83aa-2824daafcb00-host-var-lib-cni-multus\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833596 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d24e878-c4a6-4670-898b-42d157dcf067-sys-fs\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833633 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-kubelet\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833672 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-host-kubelet\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833692 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-env-overrides\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833749 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v225\" (UniqueName: \"kubernetes.io/projected/5d24e878-c4a6-4670-898b-42d157dcf067-kube-api-access-8v225\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833808 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-run\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.835007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833838 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ba4875c4-1642-481c-88b7-f59c243040e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.835646 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833874 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-run-systemd\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.835646 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833899 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-node-log\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.835646 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833914 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85439f8e-7098-47a1-ae54-d6b83a534da8-tmp\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.835646 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.833974 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-run-systemd\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.835646 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.834001 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-node-log\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.835646 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.834115 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-env-overrides\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.837314 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.837294 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-ovn-node-metrics-cert\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.851086 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.851064 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hxd7\" (UniqueName: \"kubernetes.io/projected/970378b4-f06a-43ba-857e-1e9f1ecc88cb-kube-api-access-6hxd7\") pod \"iptables-alerter-2t2vq\" (UID: \"970378b4-f06a-43ba-857e-1e9f1ecc88cb\") " pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:05.851282 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.851262 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hj4m\" (UniqueName: \"kubernetes.io/projected/f64eb380-73df-46a4-9766-dd8d99bc4625-kube-api-access-6hj4m\") pod \"node-ca-4bf69\" (UID: \"f64eb380-73df-46a4-9766-dd8d99bc4625\") " pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:05.851496 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.851464 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mmdg\" (UniqueName: \"kubernetes.io/projected/68080680-0551-487c-80ed-b93acb2f68f8-kube-api-access-8mmdg\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:05.853268 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.853244 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v225\" (UniqueName: \"kubernetes.io/projected/5d24e878-c4a6-4670-898b-42d157dcf067-kube-api-access-8v225\") pod \"aws-ebs-csi-driver-node-9fbr8\" (UID: \"5d24e878-c4a6-4670-898b-42d157dcf067\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:05.855034 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.854965 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks25k\" (UniqueName: \"kubernetes.io/projected/8118f2be-bfb1-46de-8a34-80bb58e1a5e7-kube-api-access-ks25k\") pod \"ovnkube-node-c8w2h\" (UID: \"8118f2be-bfb1-46de-8a34-80bb58e1a5e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:05.855182 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.855164 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2wjp\" (UniqueName: \"kubernetes.io/projected/277802fb-ac81-406a-83aa-2824daafcb00-kube-api-access-z2wjp\") pod \"multus-7tt2w\" (UID: \"277802fb-ac81-406a-83aa-2824daafcb00\") " pod="openshift-multus/multus-7tt2w" Sep 05 06:25:05.934854 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.934821 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/93d036de-ea2d-406d-a61d-4f6c972acc71-kubelet-config\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:05.935023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.934861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302f2dc8-13ec-46b9-b694-f66048d4e2c3-tmp-dir\") pod \"node-resolver-m42bc\" (UID: \"302f2dc8-13ec-46b9-b694-f66048d4e2c3\") " pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:05.935023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.934887 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-run\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.934910 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ba4875c4-1642-481c-88b7-f59c243040e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.935023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.934933 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85439f8e-7098-47a1-ae54-d6b83a534da8-tmp\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.934955 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-sys\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.934977 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6td\" (UniqueName: \"kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td\") pod \"network-check-target-dfdlw\" (UID: \"be8cb322-a209-49e2-bb7b-22352fcf54c0\") " pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935042 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-sys\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935111 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/93d036de-ea2d-406d-a61d-4f6c972acc71-kubelet-config\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935146 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-run\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935167 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d8ad0f2-e276-402b-8270-c6194aa5270f-agent-certs\") pod \"konnectivity-agent-kvx7p\" (UID: \"5d8ad0f2-e276-402b-8270-c6194aa5270f\") " pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/93d036de-ea2d-406d-a61d-4f6c972acc71-dbus\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935199 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-lib-modules\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935212 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-host\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935227 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302f2dc8-13ec-46b9-b694-f66048d4e2c3-tmp-dir\") pod \"node-resolver-m42bc\" (UID: \"302f2dc8-13ec-46b9-b694-f66048d4e2c3\") " pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935271 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/93d036de-ea2d-406d-a61d-4f6c972acc71-dbus\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935289 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-lib-modules\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935302 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-sysconfig\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935323 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-host\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935347 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-systemd\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935372 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba4875c4-1642-481c-88b7-f59c243040e4-cnibin\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935411 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-sysconfig\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba4875c4-1642-481c-88b7-f59c243040e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935443 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-systemd\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935457 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ba4875c4-1642-481c-88b7-f59c243040e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935460 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba4875c4-1642-481c-88b7-f59c243040e4-cnibin\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935501 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-sysctl-d\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935525 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba4875c4-1642-481c-88b7-f59c243040e4-system-cni-dir\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935548 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-kubernetes\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935564 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ba4875c4-1642-481c-88b7-f59c243040e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935571 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/302f2dc8-13ec-46b9-b694-f66048d4e2c3-hosts-file\") pod \"node-resolver-m42bc\" (UID: \"302f2dc8-13ec-46b9-b694-f66048d4e2c3\") " pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935597 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-sysctl-d\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935603 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba4875c4-1642-481c-88b7-f59c243040e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935622 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba4875c4-1642-481c-88b7-f59c243040e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935624 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/302f2dc8-13ec-46b9-b694-f66048d4e2c3-hosts-file\") pod \"node-resolver-m42bc\" (UID: \"302f2dc8-13ec-46b9-b694-f66048d4e2c3\") " pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:05.935867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935642 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba4875c4-1642-481c-88b7-f59c243040e4-system-cni-dir\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935653 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-kubernetes\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935746 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-modprobe-d\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935781 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkcv9\" (UniqueName: \"kubernetes.io/projected/302f2dc8-13ec-46b9-b694-f66048d4e2c3-kube-api-access-jkcv9\") pod \"node-resolver-m42bc\" (UID: \"302f2dc8-13ec-46b9-b694-f66048d4e2c3\") " pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935834 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-sysctl-conf\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-var-lib-kubelet\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935857 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-modprobe-d\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935885 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcr8r\" (UniqueName: \"kubernetes.io/projected/85439f8e-7098-47a1-ae54-d6b83a534da8-kube-api-access-pcr8r\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-tuned\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935938 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsxn6\" (UniqueName: \"kubernetes.io/projected/ba4875c4-1642-481c-88b7-f59c243040e4-kube-api-access-rsxn6\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.935949 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935967 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ba4875c4-1642-481c-88b7-f59c243040e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.935972 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d8ad0f2-e276-402b-8270-c6194aa5270f-konnectivity-ca\") pod \"konnectivity-agent-kvx7p\" (UID: \"5d8ad0f2-e276-402b-8270-c6194aa5270f\") " pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.936010 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret podName:93d036de-ea2d-406d-a61d-4f6c972acc71 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:06.435991741 +0000 UTC m=+3.119496628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret") pod "global-pull-secret-syncer-vcd6d" (UID: "93d036de-ea2d-406d-a61d-4f6c972acc71") : object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.936098 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba4875c4-1642-481c-88b7-f59c243040e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.936121 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-sysctl-conf\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.936338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.936169 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85439f8e-7098-47a1-ae54-d6b83a534da8-var-lib-kubelet\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.937006 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.936516 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d8ad0f2-e276-402b-8270-c6194aa5270f-konnectivity-ca\") pod \"konnectivity-agent-kvx7p\" (UID: \"5d8ad0f2-e276-402b-8270-c6194aa5270f\") " pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:05.937789 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.937747 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85439f8e-7098-47a1-ae54-d6b83a534da8-tmp\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.937943 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.937920 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d8ad0f2-e276-402b-8270-c6194aa5270f-agent-certs\") pod \"konnectivity-agent-kvx7p\" (UID: \"5d8ad0f2-e276-402b-8270-c6194aa5270f\") " pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:05.939044 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.939024 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85439f8e-7098-47a1-ae54-d6b83a534da8-etc-tuned\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.941836 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.941799 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 06:25:05.941836 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.941824 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 06:25:05.941836 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.941840 2583 projected.go:196] Error preparing data for projected volume kube-api-access-5w6td for pod openshift-network-diagnostics/network-check-target-dfdlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:05.942047 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:05.941906 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td podName:be8cb322-a209-49e2-bb7b-22352fcf54c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:06.441887426 +0000 UTC m=+3.125392322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5w6td" (UniqueName: "kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td") pod "network-check-target-dfdlw" (UID: "be8cb322-a209-49e2-bb7b-22352fcf54c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:05.945450 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.945399 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsxn6\" (UniqueName: \"kubernetes.io/projected/ba4875c4-1642-481c-88b7-f59c243040e4-kube-api-access-rsxn6\") pod \"multus-additional-cni-plugins-kglkm\" (UID: \"ba4875c4-1642-481c-88b7-f59c243040e4\") " pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:05.946002 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.945981 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcr8r\" (UniqueName: \"kubernetes.io/projected/85439f8e-7098-47a1-ae54-d6b83a534da8-kube-api-access-pcr8r\") pod \"tuned-7bj9p\" (UID: \"85439f8e-7098-47a1-ae54-d6b83a534da8\") " pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:05.946083 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.946044 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkcv9\" (UniqueName: \"kubernetes.io/projected/302f2dc8-13ec-46b9-b694-f66048d4e2c3-kube-api-access-jkcv9\") pod \"node-resolver-m42bc\" (UID: \"302f2dc8-13ec-46b9-b694-f66048d4e2c3\") " pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:05.946406 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:05.946390 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 05 06:25:06.017589 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.017551 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7tt2w" Sep 05 06:25:06.025099 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.025077 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:06.034657 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.034636 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" Sep 05 06:25:06.041183 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.041165 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2t2vq" Sep 05 06:25:06.048888 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.048856 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4bf69" Sep 05 06:25:06.055595 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.055578 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" Sep 05 06:25:06.066145 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.066127 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m42bc" Sep 05 06:25:06.073761 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.073744 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kglkm" Sep 05 06:25:06.078327 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.078312 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:06.338663 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.338590 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:06.338796 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:06.338752 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:06.338837 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:06.338813 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs podName:68080680-0551-487c-80ed-b93acb2f68f8 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:07.338799491 +0000 UTC m=+4.022304376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs") pod "network-metrics-daemon-m8fn4" (UID: "68080680-0551-487c-80ed-b93acb2f68f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:06.439745 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.439711 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:06.439917 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:06.439840 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:06.439917 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:06.439904 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret podName:93d036de-ea2d-406d-a61d-4f6c972acc71 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:07.439885811 +0000 UTC m=+4.123390710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret") pod "global-pull-secret-syncer-vcd6d" (UID: "93d036de-ea2d-406d-a61d-4f6c972acc71") : object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:06.540181 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.540143 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6td\" (UniqueName: \"kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td\") pod \"network-check-target-dfdlw\" (UID: \"be8cb322-a209-49e2-bb7b-22352fcf54c0\") " pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:06.540361 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:06.540312 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 06:25:06.540361 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:06.540338 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 06:25:06.540361 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:06.540352 2583 projected.go:196] Error preparing data for projected volume kube-api-access-5w6td for pod openshift-network-diagnostics/network-check-target-dfdlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:06.540549 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:06.540406 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td podName:be8cb322-a209-49e2-bb7b-22352fcf54c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:07.54038956 +0000 UTC m=+4.223894445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5w6td" (UniqueName: "kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td") pod "network-check-target-dfdlw" (UID: "be8cb322-a209-49e2-bb7b-22352fcf54c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:06.754225 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:06.754198 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod970378b4_f06a_43ba_857e_1e9f1ecc88cb.slice/crio-500dfb336733495002f37e87d113afd2552601efd113420d003f3c6828b37172 WatchSource:0}: Error finding container 500dfb336733495002f37e87d113afd2552601efd113420d003f3c6828b37172: Status 404 returned error can't find the container with id 500dfb336733495002f37e87d113afd2552601efd113420d003f3c6828b37172 Sep 05 06:25:06.755423 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:06.755395 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod302f2dc8_13ec_46b9_b694_f66048d4e2c3.slice/crio-4602a6df1378fb239d883a7bb8f1f7953f2f129428b6643aba9a43b8a18b78b8 WatchSource:0}: Error finding container 4602a6df1378fb239d883a7bb8f1f7953f2f129428b6643aba9a43b8a18b78b8: Status 404 returned error can't find the container with id 4602a6df1378fb239d883a7bb8f1f7953f2f129428b6643aba9a43b8a18b78b8 Sep 05 06:25:06.756383 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:06.756358 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64eb380_73df_46a4_9766_dd8d99bc4625.slice/crio-348beff8f52e873aa11b6bf8a37de4ae696afb4b2c268f72cf8f1eaaf9551006 WatchSource:0}: Error finding container 348beff8f52e873aa11b6bf8a37de4ae696afb4b2c268f72cf8f1eaaf9551006: Status 404 returned error can't find the container with id 348beff8f52e873aa11b6bf8a37de4ae696afb4b2c268f72cf8f1eaaf9551006 Sep 05 06:25:06.757452 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:06.757430 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8118f2be_bfb1_46de_8a34_80bb58e1a5e7.slice/crio-940277fd8928f48611a97d5c233916d55a8e9764c71b91ed60d91567fdb22595 WatchSource:0}: Error finding container 940277fd8928f48611a97d5c233916d55a8e9764c71b91ed60d91567fdb22595: Status 404 returned error can't find the container with id 940277fd8928f48611a97d5c233916d55a8e9764c71b91ed60d91567fdb22595 Sep 05 06:25:06.760674 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:06.760652 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277802fb_ac81_406a_83aa_2824daafcb00.slice/crio-1d33f17a4d37783a90beaeb59be769b7bd4e2f484b7febfd88413861f25914dd WatchSource:0}: Error finding container 1d33f17a4d37783a90beaeb59be769b7bd4e2f484b7febfd88413861f25914dd: Status 404 returned error can't find the container with id 1d33f17a4d37783a90beaeb59be769b7bd4e2f484b7febfd88413861f25914dd Sep 05 06:25:06.761933 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:06.761905 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4875c4_1642_481c_88b7_f59c243040e4.slice/crio-74dabfe5944dcfe30b6b4fdead12d13cbeef6cf77d2df111e603b34fd556d0cc WatchSource:0}: Error finding container 74dabfe5944dcfe30b6b4fdead12d13cbeef6cf77d2df111e603b34fd556d0cc: Status 404 returned error can't find the container with id 74dabfe5944dcfe30b6b4fdead12d13cbeef6cf77d2df111e603b34fd556d0cc Sep 05 06:25:06.763303 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:06.763284 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d24e878_c4a6_4670_898b_42d157dcf067.slice/crio-bb0a0b2cbfc7cd24e1d98067b107fb005245f00d54171fede590220b780986a4 WatchSource:0}: Error finding container bb0a0b2cbfc7cd24e1d98067b107fb005245f00d54171fede590220b780986a4: Status 404 returned error can't find the container with id bb0a0b2cbfc7cd24e1d98067b107fb005245f00d54171fede590220b780986a4 Sep 05 06:25:06.763955 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:06.763935 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85439f8e_7098_47a1_ae54_d6b83a534da8.slice/crio-1f9bc832b16725f64d6a31be6b3375353bd459225b82cd3365afb760bd150b0a WatchSource:0}: Error finding container 1f9bc832b16725f64d6a31be6b3375353bd459225b82cd3365afb760bd150b0a: Status 404 returned error can't find the container with id 1f9bc832b16725f64d6a31be6b3375353bd459225b82cd3365afb760bd150b0a Sep 05 06:25:06.765301 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.765220 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 06:20:05 +0000 UTC" deadline="2028-03-26 17:51:44.512813579 +0000 UTC" Sep 05 06:25:06.765301 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.765246 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13643h26m37.747571343s" Sep 05 06:25:06.765455 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:06.765400 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d8ad0f2_e276_402b_8270_c6194aa5270f.slice/crio-c9d4f805b36866c35715d1530cf1b351985df4aab4a053465757ad0a36370c49 WatchSource:0}: Error finding container c9d4f805b36866c35715d1530cf1b351985df4aab4a053465757ad0a36370c49: Status 404 returned error can't find the container with id c9d4f805b36866c35715d1530cf1b351985df4aab4a053465757ad0a36370c49 Sep 05 06:25:06.816769 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.816747 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:06.816874 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:06.816852 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:06.816952 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.816906 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:06.817012 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:06.816993 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:06.825497 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.825457 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" event={"ID":"85439f8e-7098-47a1-ae54-d6b83a534da8","Type":"ContainerStarted","Data":"1f9bc832b16725f64d6a31be6b3375353bd459225b82cd3365afb760bd150b0a"} Sep 05 06:25:06.826867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.826820 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kglkm" event={"ID":"ba4875c4-1642-481c-88b7-f59c243040e4","Type":"ContainerStarted","Data":"74dabfe5944dcfe30b6b4fdead12d13cbeef6cf77d2df111e603b34fd556d0cc"} Sep 05 06:25:06.828093 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.828074 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7tt2w" event={"ID":"277802fb-ac81-406a-83aa-2824daafcb00","Type":"ContainerStarted","Data":"1d33f17a4d37783a90beaeb59be769b7bd4e2f484b7febfd88413861f25914dd"} Sep 05 06:25:06.829325 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.829307 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" event={"ID":"8118f2be-bfb1-46de-8a34-80bb58e1a5e7","Type":"ContainerStarted","Data":"940277fd8928f48611a97d5c233916d55a8e9764c71b91ed60d91567fdb22595"} Sep 05 06:25:06.830559 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.830493 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2t2vq" event={"ID":"970378b4-f06a-43ba-857e-1e9f1ecc88cb","Type":"ContainerStarted","Data":"500dfb336733495002f37e87d113afd2552601efd113420d003f3c6828b37172"} Sep 05 06:25:06.831793 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.831764 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" event={"ID":"5d24e878-c4a6-4670-898b-42d157dcf067","Type":"ContainerStarted","Data":"bb0a0b2cbfc7cd24e1d98067b107fb005245f00d54171fede590220b780986a4"} Sep 05 06:25:06.832872 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.832839 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4bf69" event={"ID":"f64eb380-73df-46a4-9766-dd8d99bc4625","Type":"ContainerStarted","Data":"348beff8f52e873aa11b6bf8a37de4ae696afb4b2c268f72cf8f1eaaf9551006"} Sep 05 06:25:06.834303 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.834268 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m42bc" event={"ID":"302f2dc8-13ec-46b9-b694-f66048d4e2c3","Type":"ContainerStarted","Data":"4602a6df1378fb239d883a7bb8f1f7953f2f129428b6643aba9a43b8a18b78b8"} Sep 05 06:25:06.840170 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.840136 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kvx7p" event={"ID":"5d8ad0f2-e276-402b-8270-c6194aa5270f","Type":"ContainerStarted","Data":"c9d4f805b36866c35715d1530cf1b351985df4aab4a053465757ad0a36370c49"} Sep 05 06:25:06.851712 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:06.851665 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" podStartSLOduration=1.85165105 podStartE2EDuration="1.85165105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:06.851502379 +0000 UTC m=+3.535007285" watchObservedRunningTime="2026-09-05 06:25:06.85165105 +0000 UTC m=+3.535155957" Sep 05 06:25:07.345665 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:07.344723 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:07.345665 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:07.345183 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:07.345665 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:07.345251 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs podName:68080680-0551-487c-80ed-b93acb2f68f8 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:09.345231767 +0000 UTC m=+6.028736665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs") pod "network-metrics-daemon-m8fn4" (UID: "68080680-0551-487c-80ed-b93acb2f68f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:07.445951 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:07.445885 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:07.446135 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:07.446066 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:07.446135 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:07.446133 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret podName:93d036de-ea2d-406d-a61d-4f6c972acc71 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:09.446114175 +0000 UTC m=+6.129619074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret") pod "global-pull-secret-syncer-vcd6d" (UID: "93d036de-ea2d-406d-a61d-4f6c972acc71") : object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:07.547165 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:07.547128 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6td\" (UniqueName: \"kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td\") pod \"network-check-target-dfdlw\" (UID: \"be8cb322-a209-49e2-bb7b-22352fcf54c0\") " pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:07.547375 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:07.547358 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 06:25:07.547450 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:07.547386 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 06:25:07.547450 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:07.547401 2583 projected.go:196] Error preparing data for projected volume kube-api-access-5w6td for pod openshift-network-diagnostics/network-check-target-dfdlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:07.547584 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:07.547455 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td podName:be8cb322-a209-49e2-bb7b-22352fcf54c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:09.547438522 +0000 UTC m=+6.230943413 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5w6td" (UniqueName: "kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td") pod "network-check-target-dfdlw" (UID: "be8cb322-a209-49e2-bb7b-22352fcf54c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:07.766410 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:07.766260 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 06:20:05 +0000 UTC" deadline="2028-04-19 22:17:55.624657822 +0000 UTC" Sep 05 06:25:07.766410 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:07.766324 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14223h52m47.858337976s" Sep 05 06:25:07.817154 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:07.816699 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:07.817154 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:07.816824 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:07.853293 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:07.853257 2583 generic.go:356] "Generic (PLEG): container finished" podID="4f563eaece2c34e5022dfd42837612e6" containerID="e991c5e4ac8ca0fc4d40d7eb58f450373648c10c04ffde66622b2e65162813ba" exitCode=0 Sep 05 06:25:07.853466 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:07.853356 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" event={"ID":"4f563eaece2c34e5022dfd42837612e6","Type":"ContainerDied","Data":"e991c5e4ac8ca0fc4d40d7eb58f450373648c10c04ffde66622b2e65162813ba"} Sep 05 06:25:07.875446 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:07.875415 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-24.ec2.internal" event={"ID":"36b5895e28f6f5810ea58bd85d6e692d","Type":"ContainerStarted","Data":"3af65836dd5bb27c21b06ba2fe74a03c9039565b412b888d479b14a62588a7e5"} Sep 05 06:25:08.817056 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:08.816980 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:08.817497 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:08.817115 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:08.817638 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:08.817618 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:08.817753 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:08.817732 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:08.909364 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:08.909325 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" event={"ID":"4f563eaece2c34e5022dfd42837612e6","Type":"ContainerStarted","Data":"3202e75efe7c0ca633493a56aec4ce0f6d8770a4472cc4a9edf7450ec5ea0d12"} Sep 05 06:25:08.923630 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:08.923567 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-24.ec2.internal" podStartSLOduration=3.9235452 podStartE2EDuration="3.9235452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:08.922894746 +0000 UTC m=+5.606399666" watchObservedRunningTime="2026-09-05 06:25:08.9235452 +0000 UTC m=+5.607050107" Sep 05 06:25:09.362330 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:09.362291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:09.362526 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:09.362443 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:09.362526 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:09.362519 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs podName:68080680-0551-487c-80ed-b93acb2f68f8 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:13.362501967 +0000 UTC m=+10.046006853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs") pod "network-metrics-daemon-m8fn4" (UID: "68080680-0551-487c-80ed-b93acb2f68f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:09.462974 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:09.462936 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:09.463175 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:09.463080 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:09.463175 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:09.463140 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret podName:93d036de-ea2d-406d-a61d-4f6c972acc71 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:13.463123366 +0000 UTC m=+10.146628265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret") pod "global-pull-secret-syncer-vcd6d" (UID: "93d036de-ea2d-406d-a61d-4f6c972acc71") : object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:09.564841 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:09.564160 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6td\" (UniqueName: \"kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td\") pod \"network-check-target-dfdlw\" (UID: \"be8cb322-a209-49e2-bb7b-22352fcf54c0\") " pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:09.564841 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:09.564345 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 06:25:09.564841 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:09.564366 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 06:25:09.564841 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:09.564379 2583 projected.go:196] Error preparing data for projected volume kube-api-access-5w6td for pod openshift-network-diagnostics/network-check-target-dfdlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:09.564841 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:09.564440 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td podName:be8cb322-a209-49e2-bb7b-22352fcf54c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:13.564421385 +0000 UTC m=+10.247926269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5w6td" (UniqueName: "kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td") pod "network-check-target-dfdlw" (UID: "be8cb322-a209-49e2-bb7b-22352fcf54c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:09.818500 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:09.816642 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:09.818500 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:09.816773 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:10.816723 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:10.816527 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:10.816723 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:10.816660 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:10.816983 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:10.816759 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:10.816983 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:10.816871 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:11.816753 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:11.816715 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:11.817208 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:11.816858 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:12.816372 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:12.816340 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:12.816561 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:12.816340 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:12.816561 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:12.816459 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:12.816668 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:12.816593 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:13.399331 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:13.399274 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:13.399810 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:13.399407 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:13.399810 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:13.399472 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs podName:68080680-0551-487c-80ed-b93acb2f68f8 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:21.399450813 +0000 UTC m=+18.082955698 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs") pod "network-metrics-daemon-m8fn4" (UID: "68080680-0551-487c-80ed-b93acb2f68f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:13.500317 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:13.500281 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:13.500527 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:13.500510 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:13.500665 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:13.500576 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret podName:93d036de-ea2d-406d-a61d-4f6c972acc71 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:21.500556827 +0000 UTC m=+18.184061734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret") pod "global-pull-secret-syncer-vcd6d" (UID: "93d036de-ea2d-406d-a61d-4f6c972acc71") : object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:13.600706 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:13.600670 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6td\" (UniqueName: \"kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td\") pod \"network-check-target-dfdlw\" (UID: \"be8cb322-a209-49e2-bb7b-22352fcf54c0\") " pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:13.600899 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:13.600881 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 06:25:13.601001 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:13.600907 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 06:25:13.601001 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:13.600921 2583 projected.go:196] Error preparing data for projected volume kube-api-access-5w6td for pod openshift-network-diagnostics/network-check-target-dfdlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:13.601001 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:13.600978 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td podName:be8cb322-a209-49e2-bb7b-22352fcf54c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:21.60096024 +0000 UTC m=+18.284465126 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5w6td" (UniqueName: "kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td") pod "network-check-target-dfdlw" (UID: "be8cb322-a209-49e2-bb7b-22352fcf54c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:13.817612 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:13.817533 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:13.817765 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:13.817648 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:14.816598 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:14.816526 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:14.816598 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:14.816577 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:14.817112 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:14.816655 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:14.817112 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:14.816768 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:15.816431 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:15.816403 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:15.816593 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:15.816510 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:16.816702 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:16.816669 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:16.817075 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:16.816780 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:16.817075 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:16.816809 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:16.817075 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:16.816852 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:17.816334 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:17.816298 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:17.816556 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:17.816455 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:18.816877 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:18.816837 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:18.817316 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:18.816835 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:18.817316 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:18.816995 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:18.817316 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:18.817040 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:19.816511 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:19.816463 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:19.816674 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:19.816608 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:20.816805 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:20.816769 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:20.816805 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:20.816795 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:20.817384 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:20.816884 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:20.817384 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:20.817009 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:21.454730 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:21.454698 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:21.454891 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:21.454826 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:21.454954 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:21.454897 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs podName:68080680-0551-487c-80ed-b93acb2f68f8 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:37.454878112 +0000 UTC m=+34.138382999 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs") pod "network-metrics-daemon-m8fn4" (UID: "68080680-0551-487c-80ed-b93acb2f68f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:21.555889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:21.555855 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:21.556059 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:21.555982 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:21.556059 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:21.556050 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret podName:93d036de-ea2d-406d-a61d-4f6c972acc71 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:37.556030645 +0000 UTC m=+34.239535553 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret") pod "global-pull-secret-syncer-vcd6d" (UID: "93d036de-ea2d-406d-a61d-4f6c972acc71") : object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:21.656430 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:21.656388 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6td\" (UniqueName: \"kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td\") pod \"network-check-target-dfdlw\" (UID: \"be8cb322-a209-49e2-bb7b-22352fcf54c0\") " pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:21.656587 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:21.656558 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 06:25:21.656587 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:21.656586 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 06:25:21.656694 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:21.656600 2583 projected.go:196] Error preparing data for projected volume kube-api-access-5w6td for pod openshift-network-diagnostics/network-check-target-dfdlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:21.656694 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:21.656660 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td podName:be8cb322-a209-49e2-bb7b-22352fcf54c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:25:37.656644559 +0000 UTC m=+34.340149444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5w6td" (UniqueName: "kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td") pod "network-check-target-dfdlw" (UID: "be8cb322-a209-49e2-bb7b-22352fcf54c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:21.816179 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:21.816104 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:21.816335 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:21.816230 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:22.816578 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:22.816540 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:22.817025 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:22.816582 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:22.817025 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:22.816665 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:22.817025 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:22.816777 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:23.817304 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:23.817273 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:23.817668 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:23.817364 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:24.816387 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.816183 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:24.816468 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.816206 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:24.816468 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:24.816450 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:24.816581 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:24.816558 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:24.948172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.948144 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kvx7p" event={"ID":"5d8ad0f2-e276-402b-8270-c6194aa5270f","Type":"ContainerStarted","Data":"655ab561c6f0abc27e0336dbafb35b3dddb1749de966626a10d86f5ae2ba564e"} Sep 05 06:25:24.949694 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.949671 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" event={"ID":"85439f8e-7098-47a1-ae54-d6b83a534da8","Type":"ContainerStarted","Data":"3831c95901ea7e37cdd6ed6757daa00f2ff95e49f78c82e2c588c538803326c2"} Sep 05 06:25:24.951107 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.951083 2583 generic.go:356] "Generic (PLEG): container finished" podID="ba4875c4-1642-481c-88b7-f59c243040e4" containerID="2a00bc31b280b18291aee0d5fe6f16bb618e17e678e61886ff072fa162b5e6ed" exitCode=0 Sep 05 06:25:24.951248 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.951152 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kglkm" event={"ID":"ba4875c4-1642-481c-88b7-f59c243040e4","Type":"ContainerDied","Data":"2a00bc31b280b18291aee0d5fe6f16bb618e17e678e61886ff072fa162b5e6ed"} Sep 05 06:25:24.952643 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.952614 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7tt2w" event={"ID":"277802fb-ac81-406a-83aa-2824daafcb00","Type":"ContainerStarted","Data":"bad410a9898a497a9c4a881c54ab0857ed6a484a25f563c2fccdb62053de17f2"} Sep 05 06:25:24.955263 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.955247 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:25:24.955588 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.955566 2583 generic.go:356] "Generic (PLEG): container finished" podID="8118f2be-bfb1-46de-8a34-80bb58e1a5e7" containerID="b0ef455914d172cd0a365770ef66b4a91a217644a99c351fe2c093c57dc53144" exitCode=1 Sep 05 06:25:24.955659 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.955636 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" event={"ID":"8118f2be-bfb1-46de-8a34-80bb58e1a5e7","Type":"ContainerStarted","Data":"1a3b7c72d12b4810c912532752187646c4f13a445aaa038e4a916f968b4a3779"} Sep 05 06:25:24.955715 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.955701 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" event={"ID":"8118f2be-bfb1-46de-8a34-80bb58e1a5e7","Type":"ContainerStarted","Data":"92cd3490cd4a354eb59d287bd81b2f35dfcfeaa9f92ef59b3c8f487799ad0b5b"} Sep 05 06:25:24.955770 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.955751 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" event={"ID":"8118f2be-bfb1-46de-8a34-80bb58e1a5e7","Type":"ContainerStarted","Data":"decf0fd1d04845d6e00477c30d3d38066f8f205fcbe3962d130eb8bb26e9cc86"} Sep 05 06:25:24.955806 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.955785 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" event={"ID":"8118f2be-bfb1-46de-8a34-80bb58e1a5e7","Type":"ContainerDied","Data":"b0ef455914d172cd0a365770ef66b4a91a217644a99c351fe2c093c57dc53144"} Sep 05 06:25:24.955847 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.955814 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" event={"ID":"8118f2be-bfb1-46de-8a34-80bb58e1a5e7","Type":"ContainerStarted","Data":"cf09c88f5f674097561520171fd72380ef25f3bd8d77f12490d441e8c20720cf"} Sep 05 06:25:24.957056 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.957038 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" event={"ID":"5d24e878-c4a6-4670-898b-42d157dcf067","Type":"ContainerStarted","Data":"17d673a24c1a495010fedde6833a4c91ca772e074a78a2e38e6f1a6883bf751b"} Sep 05 06:25:24.958443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.958426 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4bf69" event={"ID":"f64eb380-73df-46a4-9766-dd8d99bc4625","Type":"ContainerStarted","Data":"9a0c630572ff6212326e66b3c17d3c36bc08e1d8f88b4113822f914f3c27ec85"} Sep 05 06:25:24.959703 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.959685 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m42bc" event={"ID":"302f2dc8-13ec-46b9-b694-f66048d4e2c3","Type":"ContainerStarted","Data":"ee6541cce55cc77cf9bf1481678ebbd00d30436c9f8ba6fd8be8b2bc5555e5dd"} Sep 05 06:25:24.964288 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.964255 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kvx7p" podStartSLOduration=10.743031972 podStartE2EDuration="20.964244812s" totalImagesPullingTime="10.22121284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:24.964013263 +0000 UTC m=+21.647518170" watchObservedRunningTime="2026-09-05 06:25:24.964244812 +0000 UTC m=+21.647749719" Sep 05 06:25:24.980467 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.980383 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7tt2w" podStartSLOduration=4.107916025 podStartE2EDuration="21.98037252s" totalImagesPullingTime="17.872456495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:24.980091065 +0000 UTC m=+21.663595975" watchObservedRunningTime="2026-09-05 06:25:24.98037252 +0000 UTC m=+21.663877426" Sep 05 06:25:24.995992 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:24.995954 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-m42bc" podStartSLOduration=3.417375202 podStartE2EDuration="20.995937271s" totalImagesPullingTime="17.578562069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:24.99568386 +0000 UTC m=+21.679188766" watchObservedRunningTime="2026-09-05 06:25:24.995937271 +0000 UTC m=+21.679442177" Sep 05 06:25:25.014354 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:25.014309 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4bf69" podStartSLOduration=4.450082177 podStartE2EDuration="22.014293408s" totalImagesPullingTime="17.564211231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:25.013995193 +0000 UTC m=+21.697500099" watchObservedRunningTime="2026-09-05 06:25:25.014293408 +0000 UTC m=+21.697798368" Sep 05 06:25:25.033307 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:25.033258 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7bj9p" podStartSLOduration=4.461763226 podStartE2EDuration="22.033239336s" totalImagesPullingTime="17.57147611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:25.032785326 +0000 UTC m=+21.716290233" watchObservedRunningTime="2026-09-05 06:25:25.033239336 +0000 UTC m=+21.716744247" Sep 05 06:25:25.816282 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:25.816247 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:25.816436 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:25.816396 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:25.851969 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:25.851936 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 05 06:25:25.965665 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:25.965585 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:25:25.966199 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:25.966167 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" event={"ID":"8118f2be-bfb1-46de-8a34-80bb58e1a5e7","Type":"ContainerStarted","Data":"df4c92a2eca5d0415ef7b4633c512d52ec0b07d5cb31ed20349e0ef98640af3f"} Sep 05 06:25:25.967884 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:25.967847 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2t2vq" event={"ID":"970378b4-f06a-43ba-857e-1e9f1ecc88cb","Type":"ContainerStarted","Data":"5f182047ab91ebfee1ec023c5fd2ad9b06d36bdfea0fb4bd9613430b414ec86a"} Sep 05 06:25:25.969926 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:25.969897 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" event={"ID":"5d24e878-c4a6-4670-898b-42d157dcf067","Type":"ContainerStarted","Data":"a9aaf10d3c5f8118b66d0c5163a27980c976d8acbd8338a4b0e84806ada8b739"} Sep 05 06:25:25.981556 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:25.981512 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2t2vq" podStartSLOduration=5.40182419 podStartE2EDuration="22.981493616s" totalImagesPullingTime="17.579669426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:25.981394779 +0000 UTC m=+22.664899687" watchObservedRunningTime="2026-09-05 06:25:25.981493616 +0000 UTC m=+22.664998518" Sep 05 06:25:26.783505 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:26.783382 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-05T06:25:25.851960342Z","UUID":"836d0a1d-110e-4afd-8507-10982fc75300","Handler":null,"Name":"","Endpoint":""} Sep 05 06:25:26.785038 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:26.785016 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 05 06:25:26.785177 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:26.785053 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 05 06:25:26.816056 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:26.816032 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:26.816191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:26.816036 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:26.816191 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:26.816141 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:26.816291 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:26.816211 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:27.594942 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:27.594894 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:27.595742 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:27.595718 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:27.816414 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:27.816385 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:27.816592 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:27.816531 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:27.976614 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:27.976304 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" event={"ID":"5d24e878-c4a6-4670-898b-42d157dcf067","Type":"ContainerStarted","Data":"d94c185f81acb29fde5b07044b6c8e30a0800e9e717705a7358692e244607d46"} Sep 05 06:25:27.980172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:27.980152 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:25:27.980603 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:27.980584 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" event={"ID":"8118f2be-bfb1-46de-8a34-80bb58e1a5e7","Type":"ContainerStarted","Data":"31b3786f964695579083aa6c06e68b1e9598d410e68f01dd0dc52dc2f99a656b"} Sep 05 06:25:27.980814 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:27.980798 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:27.981263 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:27.981244 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kvx7p" Sep 05 06:25:27.996536 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:27.996498 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9fbr8" podStartSLOduration=5.150836265 podStartE2EDuration="24.996471548s" totalImagesPullingTime="19.845635283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:27.996141949 +0000 UTC m=+24.679646858" watchObservedRunningTime="2026-09-05 06:25:27.996471548 +0000 UTC m=+24.679976454" Sep 05 06:25:28.816965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:28.816936 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:28.817386 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:28.817060 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:28.817386 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:28.817119 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:28.817386 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:28.817227 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:29.816687 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:29.816531 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:29.816784 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:29.816765 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:29.985809 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:29.985718 2583 generic.go:356] "Generic (PLEG): container finished" podID="ba4875c4-1642-481c-88b7-f59c243040e4" containerID="a202547717f6421080222570bf3fcf65e9f137e2d06e65f048658b78b4fc25fd" exitCode=0 Sep 05 06:25:29.986575 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:29.985801 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kglkm" event={"ID":"ba4875c4-1642-481c-88b7-f59c243040e4","Type":"ContainerDied","Data":"a202547717f6421080222570bf3fcf65e9f137e2d06e65f048658b78b4fc25fd"} Sep 05 06:25:29.989591 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:29.989572 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:25:29.989953 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:29.989935 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" event={"ID":"8118f2be-bfb1-46de-8a34-80bb58e1a5e7","Type":"ContainerStarted","Data":"7406d07e52f41c9f72cdade5821887ecc935ab2fc7d8b66c83158dc92205a0ec"} Sep 05 06:25:29.990293 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:29.990278 2583 scope.go:117] "RemoveContainer" containerID="b0ef455914d172cd0a365770ef66b4a91a217644a99c351fe2c093c57dc53144" Sep 05 06:25:30.816663 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:30.816644 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:30.816779 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:30.816735 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:30.816829 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:30.816788 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:30.816903 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:30.816885 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:30.994317 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:30.994237 2583 generic.go:356] "Generic (PLEG): container finished" podID="ba4875c4-1642-481c-88b7-f59c243040e4" containerID="d8637966ee98ab59c27f07dc6b7976fd2848ed758d4724cd70dcfb3f369fb329" exitCode=0 Sep 05 06:25:30.994753 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:30.994321 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kglkm" event={"ID":"ba4875c4-1642-481c-88b7-f59c243040e4","Type":"ContainerDied","Data":"d8637966ee98ab59c27f07dc6b7976fd2848ed758d4724cd70dcfb3f369fb329"} Sep 05 06:25:30.998230 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:30.998214 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:25:30.998594 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:30.998571 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" event={"ID":"8118f2be-bfb1-46de-8a34-80bb58e1a5e7","Type":"ContainerStarted","Data":"fb2d1ef0fbda2177cc79e02a5d680c78bbc4678eaa7a9e4acca8e79cfd53ea42"} Sep 05 06:25:30.998693 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:30.998638 2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 05 06:25:30.998873 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:30.998857 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:30.998943 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:30.998913 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:31.013379 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:31.013357 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:31.013795 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:31.013772 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:31.041002 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:31.040954 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" podStartSLOduration=10.443076778 podStartE2EDuration="28.040939208s" totalImagesPullingTime="17.59786243s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:31.040336868 +0000 UTC m=+27.723841776" watchObservedRunningTime="2026-09-05 06:25:31.040939208 +0000 UTC m=+27.724444114" Sep 05 06:25:31.450982 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:31.450949 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcd6d"] Sep 05 06:25:31.451164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:31.451083 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:31.451230 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:31.451193 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:31.453696 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:31.453668 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dfdlw"] Sep 05 06:25:31.453816 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:31.453755 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:31.453863 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:31.453848 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:31.454282 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:31.454261 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m8fn4"] Sep 05 06:25:31.454371 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:31.454353 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:31.454504 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:31.454460 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:32.003248 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:32.003222 2583 generic.go:356] "Generic (PLEG): container finished" podID="ba4875c4-1642-481c-88b7-f59c243040e4" containerID="9721f10e4a5b2dde395ad52ed56b78af08939b1b750792ff26617aff61d23dfc" exitCode=0 Sep 05 06:25:32.003612 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:32.003306 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kglkm" event={"ID":"ba4875c4-1642-481c-88b7-f59c243040e4","Type":"ContainerDied","Data":"9721f10e4a5b2dde395ad52ed56b78af08939b1b750792ff26617aff61d23dfc"} Sep 05 06:25:32.003612 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:32.003362 2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 05 06:25:32.816491 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:32.816233 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:32.816681 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:32.816295 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:32.816681 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:32.816605 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:32.816794 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:32.816733 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:32.868331 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:32.868304 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:25:33.816868 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:33.816845 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:33.817309 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:33.816986 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:34.019273 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:34.019222 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" podUID="8118f2be-bfb1-46de-8a34-80bb58e1a5e7" containerName="ovnkube-controller" probeResult="failure" output="" Sep 05 06:25:34.816445 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:34.816397 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:34.816649 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:34.816446 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:34.816649 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:34.816559 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:34.816763 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:34.816703 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:35.816239 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:35.816205 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:35.816772 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:35.816335 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:36.816686 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:36.816656 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:36.817240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:36.816668 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:36.817240 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:36.816751 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcd6d" podUID="93d036de-ea2d-406d-a61d-4f6c972acc71" Sep 05 06:25:36.817240 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:36.816837 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:25:37.016725 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:37.016690 2583 generic.go:356] "Generic (PLEG): container finished" podID="ba4875c4-1642-481c-88b7-f59c243040e4" containerID="e5213a273bf5ed2fcdafc692a15b018b6f4155062a391182040d4c0082043f9c" exitCode=0 Sep 05 06:25:37.016863 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:37.016747 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kglkm" event={"ID":"ba4875c4-1642-481c-88b7-f59c243040e4","Type":"ContainerDied","Data":"e5213a273bf5ed2fcdafc692a15b018b6f4155062a391182040d4c0082043f9c"} Sep 05 06:25:37.471684 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:37.471599 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:37.471820 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:37.471736 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:37.471820 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:37.471795 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs podName:68080680-0551-487c-80ed-b93acb2f68f8 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:09.471776535 +0000 UTC m=+66.155281430 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs") pod "network-metrics-daemon-m8fn4" (UID: "68080680-0551-487c-80ed-b93acb2f68f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 05 06:25:37.572576 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:37.572550 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:37.572731 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:37.572689 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:37.572773 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:37.572742 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret podName:93d036de-ea2d-406d-a61d-4f6c972acc71 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:09.572727718 +0000 UTC m=+66.256232605 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret") pod "global-pull-secret-syncer-vcd6d" (UID: "93d036de-ea2d-406d-a61d-4f6c972acc71") : object "kube-system"/"original-pull-secret" not registered Sep 05 06:25:37.673469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:37.673437 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6td\" (UniqueName: \"kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td\") pod \"network-check-target-dfdlw\" (UID: \"be8cb322-a209-49e2-bb7b-22352fcf54c0\") " pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:37.673618 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:37.673588 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 05 06:25:37.673618 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:37.673605 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 05 06:25:37.673618 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:37.673615 2583 projected.go:196] Error preparing data for projected volume kube-api-access-5w6td for pod openshift-network-diagnostics/network-check-target-dfdlw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:37.673709 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:37.673663 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td podName:be8cb322-a209-49e2-bb7b-22352fcf54c0 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:09.673647929 +0000 UTC m=+66.357152815 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5w6td" (UniqueName: "kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td") pod "network-check-target-dfdlw" (UID: "be8cb322-a209-49e2-bb7b-22352fcf54c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 05 06:25:37.816498 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:37.816413 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:37.816627 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:37.816521 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dfdlw" podUID="be8cb322-a209-49e2-bb7b-22352fcf54c0" Sep 05 06:25:38.022155 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.022126 2583 generic.go:356] "Generic (PLEG): container finished" podID="ba4875c4-1642-481c-88b7-f59c243040e4" containerID="d928909e2844bbe76a06d7bbc263f0974f9b220232a02cf4095c27e92b701796" exitCode=0 Sep 05 06:25:38.022155 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.022157 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kglkm" event={"ID":"ba4875c4-1642-481c-88b7-f59c243040e4","Type":"ContainerDied","Data":"d928909e2844bbe76a06d7bbc263f0974f9b220232a02cf4095c27e92b701796"} Sep 05 06:25:38.078331 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.078273 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-24.ec2.internal" event="NodeReady" Sep 05 06:25:38.078427 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.078396 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 05 06:25:38.121635 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.121612 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p8fq5"] Sep 05 06:25:38.137587 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.137564 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hwz4f"] Sep 05 06:25:38.138315 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.138294 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.141141 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.141115 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 05 06:25:38.141342 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.141331 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bxnqj\"" Sep 05 06:25:38.141564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.141547 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 05 06:25:38.149587 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.149570 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hwz4f"] Sep 05 06:25:38.149663 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.149596 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p8fq5"] Sep 05 06:25:38.149663 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.149660 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hwz4f" Sep 05 06:25:38.152019 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.152001 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 05 06:25:38.152120 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.152033 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 05 06:25:38.152182 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.152164 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ms7q2\"" Sep 05 06:25:38.152242 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.152230 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 05 06:25:38.278066 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.278037 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3fff948-476a-4e52-824c-62d88e038d9d-config-volume\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.278218 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.278072 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.278218 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.278098 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fff948-476a-4e52-824c-62d88e038d9d-tmp-dir\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.278218 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.278118 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vscnl\" (UniqueName: \"kubernetes.io/projected/a3fff948-476a-4e52-824c-62d88e038d9d-kube-api-access-vscnl\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.278218 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.278167 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4-cert\") pod \"ingress-canary-hwz4f\" (UID: \"2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4\") " pod="openshift-ingress-canary/ingress-canary-hwz4f" Sep 05 06:25:38.278218 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.278213 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wqrm\" (UniqueName: \"kubernetes.io/projected/2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4-kube-api-access-6wqrm\") pod \"ingress-canary-hwz4f\" (UID: \"2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4\") " pod="openshift-ingress-canary/ingress-canary-hwz4f" Sep 05 06:25:38.379161 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.379101 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4-cert\") pod \"ingress-canary-hwz4f\" (UID: \"2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4\") " pod="openshift-ingress-canary/ingress-canary-hwz4f" Sep 05 06:25:38.379161 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.379140 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wqrm\" (UniqueName: \"kubernetes.io/projected/2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4-kube-api-access-6wqrm\") pod \"ingress-canary-hwz4f\" (UID: \"2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4\") " pod="openshift-ingress-canary/ingress-canary-hwz4f" Sep 05 06:25:38.379287 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.379186 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3fff948-476a-4e52-824c-62d88e038d9d-config-volume\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.379287 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.379201 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.379287 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.379217 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fff948-476a-4e52-824c-62d88e038d9d-tmp-dir\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.379287 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.379231 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vscnl\" (UniqueName: \"kubernetes.io/projected/a3fff948-476a-4e52-824c-62d88e038d9d-kube-api-access-vscnl\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.379449 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:38.379336 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 06:25:38.379449 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:38.379399 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls podName:a3fff948-476a-4e52-824c-62d88e038d9d nodeName:}" failed. No retries permitted until 2026-09-05 06:25:38.879380461 +0000 UTC m=+35.562885353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls") pod "dns-default-p8fq5" (UID: "a3fff948-476a-4e52-824c-62d88e038d9d") : secret "dns-default-metrics-tls" not found Sep 05 06:25:38.379657 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.379637 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fff948-476a-4e52-824c-62d88e038d9d-tmp-dir\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.379844 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.379819 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3fff948-476a-4e52-824c-62d88e038d9d-config-volume\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.383509 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.383468 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4-cert\") pod \"ingress-canary-hwz4f\" (UID: \"2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4\") " pod="openshift-ingress-canary/ingress-canary-hwz4f" Sep 05 06:25:38.388129 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.388102 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wqrm\" (UniqueName: \"kubernetes.io/projected/2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4-kube-api-access-6wqrm\") pod \"ingress-canary-hwz4f\" (UID: \"2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4\") " pod="openshift-ingress-canary/ingress-canary-hwz4f" Sep 05 06:25:38.388215 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.388143 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vscnl\" (UniqueName: \"kubernetes.io/projected/a3fff948-476a-4e52-824c-62d88e038d9d-kube-api-access-vscnl\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.458312 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.458284 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hwz4f" Sep 05 06:25:38.779596 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.779515 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hwz4f"] Sep 05 06:25:38.783377 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:25:38.783353 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2582be7e_4f6c_4fdd_b852_2bed0a6d8cf4.slice/crio-3a48c80dc7ddfeea031be4022882d4d9c5aedeebef3b65d802596a1f88457c81 WatchSource:0}: Error finding container 3a48c80dc7ddfeea031be4022882d4d9c5aedeebef3b65d802596a1f88457c81: Status 404 returned error can't find the container with id 3a48c80dc7ddfeea031be4022882d4d9c5aedeebef3b65d802596a1f88457c81 Sep 05 06:25:38.816685 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.816664 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:25:38.816843 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.816824 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:25:38.819689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.819671 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 05 06:25:38.819781 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.819671 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 05 06:25:38.819781 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.819671 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6v7qh\"" Sep 05 06:25:38.881162 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:38.881137 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:38.881289 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:38.881273 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 06:25:38.881337 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:38.881330 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls podName:a3fff948-476a-4e52-824c-62d88e038d9d nodeName:}" failed. No retries permitted until 2026-09-05 06:25:39.881314366 +0000 UTC m=+36.564819251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls") pod "dns-default-p8fq5" (UID: "a3fff948-476a-4e52-824c-62d88e038d9d") : secret "dns-default-metrics-tls" not found Sep 05 06:25:39.028552 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:39.028526 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kglkm" event={"ID":"ba4875c4-1642-481c-88b7-f59c243040e4","Type":"ContainerStarted","Data":"a9422dca5fa36a51092fbeb56c1c08ec484e0fb29460e2dc196c1d8861b5f6ab"} Sep 05 06:25:39.029658 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:39.029607 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hwz4f" event={"ID":"2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4","Type":"ContainerStarted","Data":"3a48c80dc7ddfeea031be4022882d4d9c5aedeebef3b65d802596a1f88457c81"} Sep 05 06:25:39.050922 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:39.050880 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kglkm" podStartSLOduration=7.020605583 podStartE2EDuration="35.050864958s" totalImagesPullingTime="28.030259375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:39.049905061 +0000 UTC m=+35.733409969" watchObservedRunningTime="2026-09-05 06:25:39.050864958 +0000 UTC m=+35.734369867" Sep 05 06:25:39.816360 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:39.816327 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:25:39.820098 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:39.819843 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 05 06:25:39.820098 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:39.819845 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 05 06:25:39.820706 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:39.820684 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gt2zc\"" Sep 05 06:25:39.887291 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:39.887258 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:39.887430 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:39.887374 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 06:25:39.887487 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:39.887443 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls podName:a3fff948-476a-4e52-824c-62d88e038d9d nodeName:}" failed. No retries permitted until 2026-09-05 06:25:41.887424057 +0000 UTC m=+38.570928955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls") pod "dns-default-p8fq5" (UID: "a3fff948-476a-4e52-824c-62d88e038d9d") : secret "dns-default-metrics-tls" not found Sep 05 06:25:41.902695 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:41.902450 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:41.902695 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:41.902595 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 06:25:41.903270 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:41.902742 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls podName:a3fff948-476a-4e52-824c-62d88e038d9d nodeName:}" failed. No retries permitted until 2026-09-05 06:25:45.902724443 +0000 UTC m=+42.586229328 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls") pod "dns-default-p8fq5" (UID: "a3fff948-476a-4e52-824c-62d88e038d9d") : secret "dns-default-metrics-tls" not found Sep 05 06:25:42.039070 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:42.039035 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hwz4f" event={"ID":"2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4","Type":"ContainerStarted","Data":"aa2d6856c4776e249ba9acaf7f479bf158761b7c3159b368dda10591d3883884"} Sep 05 06:25:42.059979 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:42.059934 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hwz4f" podStartSLOduration=1.245206632 podStartE2EDuration="4.059916585s" totalImagesPullingTime="2.814709953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:25:42.058854508 +0000 UTC m=+38.742359438" watchObservedRunningTime="2026-09-05 06:25:42.059916585 +0000 UTC m=+38.743421494" Sep 05 06:25:45.931307 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:45.931270 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:45.931803 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:45.931375 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 06:25:45.931803 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:45.931427 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls podName:a3fff948-476a-4e52-824c-62d88e038d9d nodeName:}" failed. No retries permitted until 2026-09-05 06:25:53.931414886 +0000 UTC m=+50.614919771 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls") pod "dns-default-p8fq5" (UID: "a3fff948-476a-4e52-824c-62d88e038d9d") : secret "dns-default-metrics-tls" not found Sep 05 06:25:53.987768 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:25:53.987727 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:25:53.988209 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:53.987843 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 06:25:53.988209 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:25:53.987891 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls podName:a3fff948-476a-4e52-824c-62d88e038d9d nodeName:}" failed. No retries permitted until 2026-09-05 06:26:09.987879577 +0000 UTC m=+66.671384462 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls") pod "dns-default-p8fq5" (UID: "a3fff948-476a-4e52-824c-62d88e038d9d") : secret "dns-default-metrics-tls" not found Sep 05 06:26:04.016833 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:04.016805 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c8w2h" Sep 05 06:26:09.490754 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.490717 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:26:09.493303 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.493285 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 05 06:26:09.501207 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:26:09.501189 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 05 06:26:09.501270 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:26:09.501251 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs podName:68080680-0551-487c-80ed-b93acb2f68f8 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:13.501234263 +0000 UTC m=+130.184739147 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs") pod "network-metrics-daemon-m8fn4" (UID: "68080680-0551-487c-80ed-b93acb2f68f8") : secret "metrics-daemon-secret" not found Sep 05 06:26:09.591888 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.591864 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:26:09.594690 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.594673 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 05 06:26:09.605980 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.605963 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93d036de-ea2d-406d-a61d-4f6c972acc71-original-pull-secret\") pod \"global-pull-secret-syncer-vcd6d\" (UID: \"93d036de-ea2d-406d-a61d-4f6c972acc71\") " pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:26:09.692424 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.692383 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6td\" (UniqueName: \"kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td\") pod \"network-check-target-dfdlw\" (UID: \"be8cb322-a209-49e2-bb7b-22352fcf54c0\") " pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:26:09.695164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.695148 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 05 06:26:09.705738 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.705714 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 05 06:26:09.716246 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.716218 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w6td\" (UniqueName: \"kubernetes.io/projected/be8cb322-a209-49e2-bb7b-22352fcf54c0-kube-api-access-5w6td\") pod \"network-check-target-dfdlw\" (UID: \"be8cb322-a209-49e2-bb7b-22352fcf54c0\") " pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:26:09.730793 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.730771 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcd6d" Sep 05 06:26:09.830224 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.830195 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gt2zc\"" Sep 05 06:26:09.838052 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.838036 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:26:09.994770 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:09.994739 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:26:09.994909 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:26:09.994888 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 06:26:09.994967 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:26:09.994958 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls podName:a3fff948-476a-4e52-824c-62d88e038d9d nodeName:}" failed. No retries permitted until 2026-09-05 06:26:41.994940435 +0000 UTC m=+98.678445320 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls") pod "dns-default-p8fq5" (UID: "a3fff948-476a-4e52-824c-62d88e038d9d") : secret "dns-default-metrics-tls" not found Sep 05 06:26:10.004151 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:10.004125 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcd6d"] Sep 05 06:26:10.007442 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:26:10.007421 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d036de_ea2d_406d_a61d_4f6c972acc71.slice/crio-aef1173b4eafe81ad35c80e19e9b1c8c07ca798b435419a6b8b6b5ef70c6335b WatchSource:0}: Error finding container aef1173b4eafe81ad35c80e19e9b1c8c07ca798b435419a6b8b6b5ef70c6335b: Status 404 returned error can't find the container with id aef1173b4eafe81ad35c80e19e9b1c8c07ca798b435419a6b8b6b5ef70c6335b Sep 05 06:26:10.105743 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:10.105678 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dfdlw"] Sep 05 06:26:10.107956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:10.107934 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcd6d" event={"ID":"93d036de-ea2d-406d-a61d-4f6c972acc71","Type":"ContainerStarted","Data":"aef1173b4eafe81ad35c80e19e9b1c8c07ca798b435419a6b8b6b5ef70c6335b"} Sep 05 06:26:10.109850 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:26:10.109828 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8cb322_a209_49e2_bb7b_22352fcf54c0.slice/crio-398c2271dba3c0fcc4e303ee152a5368e530ad6178604aa3aaa31e041af75914 WatchSource:0}: Error finding container 398c2271dba3c0fcc4e303ee152a5368e530ad6178604aa3aaa31e041af75914: Status 404 returned error can't find the container with id 398c2271dba3c0fcc4e303ee152a5368e530ad6178604aa3aaa31e041af75914 Sep 05 06:26:11.112384 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:11.112351 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dfdlw" event={"ID":"be8cb322-a209-49e2-bb7b-22352fcf54c0","Type":"ContainerStarted","Data":"398c2271dba3c0fcc4e303ee152a5368e530ad6178604aa3aaa31e041af75914"} Sep 05 06:26:15.124796 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:15.124750 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcd6d" event={"ID":"93d036de-ea2d-406d-a61d-4f6c972acc71","Type":"ContainerStarted","Data":"08fb87a4ddbc7a05412a7df6c3aefcb34980645c94baf982652c7fe359061517"} Sep 05 06:26:15.126243 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:15.126219 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dfdlw" event={"ID":"be8cb322-a209-49e2-bb7b-22352fcf54c0","Type":"ContainerStarted","Data":"620f95bfca58a26c038f92152a52186b2920eec2ecf7051be95f290e10412b27"} Sep 05 06:26:15.126364 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:15.126347 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:26:15.143684 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:15.143643 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vcd6d" podStartSLOduration=66.793900892 podStartE2EDuration="1m11.143630732s" totalImagesPullingTime="4.34972984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:15.142746881 +0000 UTC m=+71.826251785" watchObservedRunningTime="2026-09-05 06:26:15.143630732 +0000 UTC m=+71.827135618" Sep 05 06:26:15.162145 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:15.162105 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dfdlw" podStartSLOduration=66.918041945 podStartE2EDuration="1m11.162092857s" totalImagesPullingTime="4.244050912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:15.161648553 +0000 UTC m=+71.845153459" watchObservedRunningTime="2026-09-05 06:26:15.162092857 +0000 UTC m=+71.845597764" Sep 05 06:26:42.000691 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:42.000645 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:26:42.001131 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:26:42.000761 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 05 06:26:42.001131 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:26:42.000812 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls podName:a3fff948-476a-4e52-824c-62d88e038d9d nodeName:}" failed. No retries permitted until 2026-09-05 06:27:46.000798525 +0000 UTC m=+162.684303432 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls") pod "dns-default-p8fq5" (UID: "a3fff948-476a-4e52-824c-62d88e038d9d") : secret "dns-default-metrics-tls" not found Sep 05 06:26:46.131199 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:26:46.131169 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dfdlw" Sep 05 06:27:13.593443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:13.593384 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:27:13.593950 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:13.593568 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 05 06:27:13.593950 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:13.593644 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs podName:68080680-0551-487c-80ed-b93acb2f68f8 nodeName:}" failed. No retries permitted until 2026-09-05 06:29:15.593624831 +0000 UTC m=+252.277129720 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs") pod "network-metrics-daemon-m8fn4" (UID: "68080680-0551-487c-80ed-b93acb2f68f8") : secret "metrics-daemon-secret" not found Sep 05 06:27:15.542010 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.541976 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml"] Sep 05 06:27:15.543652 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.543636 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:15.547399 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.547367 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 05 06:27:15.547587 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.547373 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 05 06:27:15.548770 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.548746 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qmwmv\"" Sep 05 06:27:15.548869 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.548794 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 05 06:27:15.548869 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.548821 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 05 06:27:15.558557 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.558538 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml"] Sep 05 06:27:15.606853 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.606828 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8h9g\" (UniqueName: \"kubernetes.io/projected/2c15fd53-f580-45ab-aacf-5a21e7500817-kube-api-access-d8h9g\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:15.606946 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.606862 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2c15fd53-f580-45ab-aacf-5a21e7500817-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:15.606946 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.606882 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:15.707492 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.707445 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8h9g\" (UniqueName: \"kubernetes.io/projected/2c15fd53-f580-45ab-aacf-5a21e7500817-kube-api-access-d8h9g\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:15.707602 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.707508 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2c15fd53-f580-45ab-aacf-5a21e7500817-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:15.707602 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.707571 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:15.707718 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:15.707704 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 05 06:27:15.707775 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:15.707766 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls podName:2c15fd53-f580-45ab-aacf-5a21e7500817 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:16.207751838 +0000 UTC m=+132.891256723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6r7ml" (UID: "2c15fd53-f580-45ab-aacf-5a21e7500817") : secret "cluster-monitoring-operator-tls" not found Sep 05 06:27:15.708095 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.708078 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2c15fd53-f580-45ab-aacf-5a21e7500817-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:15.717866 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.717842 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8h9g\" (UniqueName: \"kubernetes.io/projected/2c15fd53-f580-45ab-aacf-5a21e7500817-kube-api-access-d8h9g\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:15.745888 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.745867 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5"] Sep 05 06:27:15.747459 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.747445 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5" Sep 05 06:27:15.748930 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.748909 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5"] Sep 05 06:27:15.750319 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.750300 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-vssj4\"" Sep 05 06:27:15.750446 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.750429 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-8bx7z"] Sep 05 06:27:15.750617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.750596 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:15.750852 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.750604 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 05 06:27:15.751122 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.751025 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 05 06:27:15.753801 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.753776 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 05 06:27:15.753953 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.753934 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 05 06:27:15.754043 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.754009 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 05 06:27:15.754093 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.754069 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 05 06:27:15.754211 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.754193 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-64p9p\"" Sep 05 06:27:15.755372 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.755352 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.757672 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.757640 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 05 06:27:15.757931 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.757915 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 05 06:27:15.758068 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.758054 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 05 06:27:15.758337 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.758317 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 05 06:27:15.758592 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.758571 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9zgcq\"" Sep 05 06:27:15.762108 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.762090 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5"] Sep 05 06:27:15.763640 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.763619 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5"] Sep 05 06:27:15.765250 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.765083 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 05 06:27:15.773510 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.773491 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-8bx7z"] Sep 05 06:27:15.808871 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.808808 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad94c7a8-4641-4f4d-b7fa-56e32c098ee8-config\") pod \"service-ca-operator-859f94c4b-nj2m5\" (UID: \"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:15.808871 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.808839 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-trusted-ca\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.809007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.808877 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b89x5\" (UniqueName: \"kubernetes.io/projected/065e72da-226b-42f0-a35d-def84ac2743a-kube-api-access-b89x5\") pod \"volume-data-source-validator-695c895bd6-4nzw5\" (UID: \"065e72da-226b-42f0-a35d-def84ac2743a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5" Sep 05 06:27:15.809007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.808894 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-serving-cert\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.809007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.808912 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-config\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.809007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.808990 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvl4p\" (UniqueName: \"kubernetes.io/projected/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-kube-api-access-qvl4p\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.809131 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.809015 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad94c7a8-4641-4f4d-b7fa-56e32c098ee8-serving-cert\") pod \"service-ca-operator-859f94c4b-nj2m5\" (UID: \"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:15.809131 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.809056 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gz2g\" (UniqueName: \"kubernetes.io/projected/ad94c7a8-4641-4f4d-b7fa-56e32c098ee8-kube-api-access-8gz2g\") pod \"service-ca-operator-859f94c4b-nj2m5\" (UID: \"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:15.910163 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.910130 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b89x5\" (UniqueName: \"kubernetes.io/projected/065e72da-226b-42f0-a35d-def84ac2743a-kube-api-access-b89x5\") pod \"volume-data-source-validator-695c895bd6-4nzw5\" (UID: \"065e72da-226b-42f0-a35d-def84ac2743a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5" Sep 05 06:27:15.910163 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.910164 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-serving-cert\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.910377 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.910181 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-config\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.910377 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.910209 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvl4p\" (UniqueName: \"kubernetes.io/projected/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-kube-api-access-qvl4p\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.910377 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.910226 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad94c7a8-4641-4f4d-b7fa-56e32c098ee8-serving-cert\") pod \"service-ca-operator-859f94c4b-nj2m5\" (UID: \"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:15.910377 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.910258 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gz2g\" (UniqueName: \"kubernetes.io/projected/ad94c7a8-4641-4f4d-b7fa-56e32c098ee8-kube-api-access-8gz2g\") pod \"service-ca-operator-859f94c4b-nj2m5\" (UID: \"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:15.910377 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.910281 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad94c7a8-4641-4f4d-b7fa-56e32c098ee8-config\") pod \"service-ca-operator-859f94c4b-nj2m5\" (UID: \"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:15.910377 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.910304 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-trusted-ca\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.910980 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.910941 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-config\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.910980 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.910944 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad94c7a8-4641-4f4d-b7fa-56e32c098ee8-config\") pod \"service-ca-operator-859f94c4b-nj2m5\" (UID: \"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:15.911161 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.911090 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-trusted-ca\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.912539 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.912517 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad94c7a8-4641-4f4d-b7fa-56e32c098ee8-serving-cert\") pod \"service-ca-operator-859f94c4b-nj2m5\" (UID: \"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:15.912661 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.912627 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-serving-cert\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.918245 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.918220 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gz2g\" (UniqueName: \"kubernetes.io/projected/ad94c7a8-4641-4f4d-b7fa-56e32c098ee8-kube-api-access-8gz2g\") pod \"service-ca-operator-859f94c4b-nj2m5\" (UID: \"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:15.919195 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.919177 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvl4p\" (UniqueName: \"kubernetes.io/projected/a6b9ff6f-b0f0-43eb-8516-1939b0a95609-kube-api-access-qvl4p\") pod \"console-operator-85b454874b-8bx7z\" (UID: \"a6b9ff6f-b0f0-43eb-8516-1939b0a95609\") " pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:15.919359 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:15.919339 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b89x5\" (UniqueName: \"kubernetes.io/projected/065e72da-226b-42f0-a35d-def84ac2743a-kube-api-access-b89x5\") pod \"volume-data-source-validator-695c895bd6-4nzw5\" (UID: \"065e72da-226b-42f0-a35d-def84ac2743a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5" Sep 05 06:27:16.061773 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:16.061686 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5" Sep 05 06:27:16.070106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:16.069955 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" Sep 05 06:27:16.073958 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:16.073936 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:16.214492 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:16.214454 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:16.214659 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:16.214584 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 05 06:27:16.214659 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:16.214650 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls podName:2c15fd53-f580-45ab-aacf-5a21e7500817 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:17.214635205 +0000 UTC m=+133.898140094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6r7ml" (UID: "2c15fd53-f580-45ab-aacf-5a21e7500817") : secret "cluster-monitoring-operator-tls" not found Sep 05 06:27:16.336226 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:16.336147 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5"] Sep 05 06:27:16.339472 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:16.339428 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod065e72da_226b_42f0_a35d_def84ac2743a.slice/crio-6d98a8315b070ececed2b99d98da37b3c5a05d2f7fd9db2ced1305a00c12b948 WatchSource:0}: Error finding container 6d98a8315b070ececed2b99d98da37b3c5a05d2f7fd9db2ced1305a00c12b948: Status 404 returned error can't find the container with id 6d98a8315b070ececed2b99d98da37b3c5a05d2f7fd9db2ced1305a00c12b948 Sep 05 06:27:16.341910 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:16.341852 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5"] Sep 05 06:27:16.344737 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:16.344690 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad94c7a8_4641_4f4d_b7fa_56e32c098ee8.slice/crio-51d6a108fc39e401bfb2fbdedfb72115b417bc2f1d64e01f1e49d897d0a66903 WatchSource:0}: Error finding container 51d6a108fc39e401bfb2fbdedfb72115b417bc2f1d64e01f1e49d897d0a66903: Status 404 returned error can't find the container with id 51d6a108fc39e401bfb2fbdedfb72115b417bc2f1d64e01f1e49d897d0a66903 Sep 05 06:27:16.357255 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:16.357233 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-8bx7z"] Sep 05 06:27:16.359563 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:16.359541 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b9ff6f_b0f0_43eb_8516_1939b0a95609.slice/crio-e8e5f9ced5c1e41f6d37d86d8632f55c4d55f88167883d5f9e9ddaa0d22f2473 WatchSource:0}: Error finding container e8e5f9ced5c1e41f6d37d86d8632f55c4d55f88167883d5f9e9ddaa0d22f2473: Status 404 returned error can't find the container with id e8e5f9ced5c1e41f6d37d86d8632f55c4d55f88167883d5f9e9ddaa0d22f2473 Sep 05 06:27:17.221109 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:17.221067 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:17.221580 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:17.221244 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 05 06:27:17.221580 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:17.221323 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls podName:2c15fd53-f580-45ab-aacf-5a21e7500817 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:19.221300778 +0000 UTC m=+135.904805668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6r7ml" (UID: "2c15fd53-f580-45ab-aacf-5a21e7500817") : secret "cluster-monitoring-operator-tls" not found Sep 05 06:27:17.281364 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:17.281325 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" event={"ID":"a6b9ff6f-b0f0-43eb-8516-1939b0a95609","Type":"ContainerStarted","Data":"e8e5f9ced5c1e41f6d37d86d8632f55c4d55f88167883d5f9e9ddaa0d22f2473"} Sep 05 06:27:17.283041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:17.282961 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" event={"ID":"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8","Type":"ContainerStarted","Data":"51d6a108fc39e401bfb2fbdedfb72115b417bc2f1d64e01f1e49d897d0a66903"} Sep 05 06:27:17.285032 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:17.284955 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5" event={"ID":"065e72da-226b-42f0-a35d-def84ac2743a","Type":"ContainerStarted","Data":"6d98a8315b070ececed2b99d98da37b3c5a05d2f7fd9db2ced1305a00c12b948"} Sep 05 06:27:18.293071 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:18.293030 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5" event={"ID":"065e72da-226b-42f0-a35d-def84ac2743a","Type":"ContainerStarted","Data":"4f223899ce18645cb315edfd4aaaecbf8bddeb70259406db28d9f8dd1c96e810"} Sep 05 06:27:18.309369 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:18.309323 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-4nzw5" podStartSLOduration=2.115962272 podStartE2EDuration="3.30930883s" totalImagesPullingTime="1.193346558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:18.309023385 +0000 UTC m=+134.992528292" watchObservedRunningTime="2026-09-05 06:27:18.30930883 +0000 UTC m=+134.992813736" Sep 05 06:27:19.237716 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:19.237684 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:19.237869 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:19.237790 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 05 06:27:19.237869 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:19.237841 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls podName:2c15fd53-f580-45ab-aacf-5a21e7500817 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:23.237828062 +0000 UTC m=+139.921332946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6r7ml" (UID: "2c15fd53-f580-45ab-aacf-5a21e7500817") : secret "cluster-monitoring-operator-tls" not found Sep 05 06:27:19.297842 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:19.297814 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/0.log" Sep 05 06:27:19.298247 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:19.297853 2583 generic.go:356] "Generic (PLEG): container finished" podID="a6b9ff6f-b0f0-43eb-8516-1939b0a95609" containerID="4fc490a61c96b7534c1d3d6fa552178935bc07570e0570cc9e2101b6e73525c0" exitCode=255 Sep 05 06:27:19.298247 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:19.297912 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" event={"ID":"a6b9ff6f-b0f0-43eb-8516-1939b0a95609","Type":"ContainerDied","Data":"4fc490a61c96b7534c1d3d6fa552178935bc07570e0570cc9e2101b6e73525c0"} Sep 05 06:27:19.298247 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:19.298124 2583 scope.go:117] "RemoveContainer" containerID="4fc490a61c96b7534c1d3d6fa552178935bc07570e0570cc9e2101b6e73525c0" Sep 05 06:27:19.299540 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:19.299507 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" event={"ID":"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8","Type":"ContainerStarted","Data":"cc4270c3c3fad2dac401f5b76110e786e6b8c7f58a7f838e2af60f8be31bcd8e"} Sep 05 06:27:19.331451 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:19.331413 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" podStartSLOduration=1.84098234 podStartE2EDuration="4.331399478s" totalImagesPullingTime="2.490417138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:19.330803649 +0000 UTC m=+136.014308559" watchObservedRunningTime="2026-09-05 06:27:19.331399478 +0000 UTC m=+136.014904384" Sep 05 06:27:20.304414 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.304387 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/1.log" Sep 05 06:27:20.304836 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.304818 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/0.log" Sep 05 06:27:20.304902 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.304848 2583 generic.go:356] "Generic (PLEG): container finished" podID="a6b9ff6f-b0f0-43eb-8516-1939b0a95609" containerID="1012eba5de6ab6e8cbe316a4fdb36a1ea5dd24f137d09148090cbd82331206d9" exitCode=255 Sep 05 06:27:20.304955 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.304887 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" event={"ID":"a6b9ff6f-b0f0-43eb-8516-1939b0a95609","Type":"ContainerDied","Data":"1012eba5de6ab6e8cbe316a4fdb36a1ea5dd24f137d09148090cbd82331206d9"} Sep 05 06:27:20.305003 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.304962 2583 scope.go:117] "RemoveContainer" containerID="4fc490a61c96b7534c1d3d6fa552178935bc07570e0570cc9e2101b6e73525c0" Sep 05 06:27:20.305345 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.305319 2583 scope.go:117] "RemoveContainer" containerID="1012eba5de6ab6e8cbe316a4fdb36a1ea5dd24f137d09148090cbd82331206d9" Sep 05 06:27:20.305546 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:20.305525 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-8bx7z_openshift-console-operator(a6b9ff6f-b0f0-43eb-8516-1939b0a95609)\"" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" podUID="a6b9ff6f-b0f0-43eb-8516-1939b0a95609" Sep 05 06:27:20.477590 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.477556 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-b29nx"] Sep 05 06:27:20.481555 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.481538 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.483872 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.483849 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qtk85\"" Sep 05 06:27:20.484102 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.484080 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 05 06:27:20.484320 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.484298 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 05 06:27:20.484320 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.484306 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 05 06:27:20.484439 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.484355 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 05 06:27:20.492529 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.492503 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b29nx"] Sep 05 06:27:20.545927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.545900 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.545927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.545927 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcdw6\" (UniqueName: \"kubernetes.io/projected/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-api-access-zcdw6\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.546078 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.546015 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49f311ef-2d4d-4cf1-93e4-4138b14525a5-crio-socket\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.546078 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.546065 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49f311ef-2d4d-4cf1-93e4-4138b14525a5-data-volume\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.546161 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.546082 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.647327 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.647260 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49f311ef-2d4d-4cf1-93e4-4138b14525a5-crio-socket\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.647327 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.647306 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49f311ef-2d4d-4cf1-93e4-4138b14525a5-data-volume\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.647466 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.647378 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49f311ef-2d4d-4cf1-93e4-4138b14525a5-crio-socket\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.647466 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.647423 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.647564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.647465 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.647564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.647502 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcdw6\" (UniqueName: \"kubernetes.io/projected/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-api-access-zcdw6\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.647635 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:20.647581 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 05 06:27:20.647635 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:20.647615 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 05 06:27:20.647635 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.647628 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49f311ef-2d4d-4cf1-93e4-4138b14525a5-data-volume\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:20.647744 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:20.647640 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls podName:49f311ef-2d4d-4cf1-93e4-4138b14525a5 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:21.147625955 +0000 UTC m=+137.831130841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b29nx" (UID: "49f311ef-2d4d-4cf1-93e4-4138b14525a5") : secret "insights-runtime-extractor-tls" not found Sep 05 06:27:20.647744 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:20.647682 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm podName:49f311ef-2d4d-4cf1-93e4-4138b14525a5 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:21.147659957 +0000 UTC m=+137.831164845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b29nx" (UID: "49f311ef-2d4d-4cf1-93e4-4138b14525a5") : configmap "kube-rbac-proxy" not found Sep 05 06:27:20.658857 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:20.658828 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcdw6\" (UniqueName: \"kubernetes.io/projected/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-api-access-zcdw6\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:21.151185 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.151147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:21.151185 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.151190 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:21.151444 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:21.151267 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 05 06:27:21.151444 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:21.151307 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 05 06:27:21.151444 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:21.151327 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm podName:49f311ef-2d4d-4cf1-93e4-4138b14525a5 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:22.151309779 +0000 UTC m=+138.834814675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b29nx" (UID: "49f311ef-2d4d-4cf1-93e4-4138b14525a5") : configmap "kube-rbac-proxy" not found Sep 05 06:27:21.151444 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:21.151370 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls podName:49f311ef-2d4d-4cf1-93e4-4138b14525a5 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:22.151353425 +0000 UTC m=+138.834858322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b29nx" (UID: "49f311ef-2d4d-4cf1-93e4-4138b14525a5") : secret "insights-runtime-extractor-tls" not found Sep 05 06:27:21.308916 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.308890 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/1.log" Sep 05 06:27:21.309266 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.309244 2583 scope.go:117] "RemoveContainer" containerID="1012eba5de6ab6e8cbe316a4fdb36a1ea5dd24f137d09148090cbd82331206d9" Sep 05 06:27:21.309434 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:21.309414 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-8bx7z_openshift-console-operator(a6b9ff6f-b0f0-43eb-8516-1939b0a95609)\"" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" podUID="a6b9ff6f-b0f0-43eb-8516-1939b0a95609" Sep 05 06:27:21.650852 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.650818 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb"] Sep 05 06:27:21.654119 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.654103 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb" Sep 05 06:27:21.657205 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.657185 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 05 06:27:21.657312 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.657288 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 05 06:27:21.657405 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.657387 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-fdr4l\"" Sep 05 06:27:21.662661 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.662640 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb"] Sep 05 06:27:21.755754 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.755728 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gzgd\" (UniqueName: \"kubernetes.io/projected/5710c05e-19b5-486d-8697-1cde26e40a86-kube-api-access-2gzgd\") pod \"migrator-fdbb8fdff-4f2jb\" (UID: \"5710c05e-19b5-486d-8697-1cde26e40a86\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb" Sep 05 06:27:21.856398 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.856376 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gzgd\" (UniqueName: \"kubernetes.io/projected/5710c05e-19b5-486d-8697-1cde26e40a86-kube-api-access-2gzgd\") pod \"migrator-fdbb8fdff-4f2jb\" (UID: \"5710c05e-19b5-486d-8697-1cde26e40a86\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb" Sep 05 06:27:21.865466 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.865445 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gzgd\" (UniqueName: \"kubernetes.io/projected/5710c05e-19b5-486d-8697-1cde26e40a86-kube-api-access-2gzgd\") pod \"migrator-fdbb8fdff-4f2jb\" (UID: \"5710c05e-19b5-486d-8697-1cde26e40a86\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb" Sep 05 06:27:21.963724 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:21.963661 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb" Sep 05 06:27:22.158490 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.158437 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:22.158628 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.158493 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:22.158628 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:22.158601 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 05 06:27:22.158628 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:22.158619 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 05 06:27:22.158738 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:22.158666 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls podName:49f311ef-2d4d-4cf1-93e4-4138b14525a5 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:24.158649264 +0000 UTC m=+140.842154149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b29nx" (UID: "49f311ef-2d4d-4cf1-93e4-4138b14525a5") : secret "insights-runtime-extractor-tls" not found Sep 05 06:27:22.158738 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:22.158680 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm podName:49f311ef-2d4d-4cf1-93e4-4138b14525a5 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:24.158674246 +0000 UTC m=+140.842179131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b29nx" (UID: "49f311ef-2d4d-4cf1-93e4-4138b14525a5") : configmap "kube-rbac-proxy" not found Sep 05 06:27:22.228970 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.228904 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb"] Sep 05 06:27:22.231913 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:22.231890 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5710c05e_19b5_486d_8697_1cde26e40a86.slice/crio-cd067b21c8f0e5d3259bc06b52c629ecf3940d707eca135010206a425d38d4bb WatchSource:0}: Error finding container cd067b21c8f0e5d3259bc06b52c629ecf3940d707eca135010206a425d38d4bb: Status 404 returned error can't find the container with id cd067b21c8f0e5d3259bc06b52c629ecf3940d707eca135010206a425d38d4bb Sep 05 06:27:22.313541 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.313511 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb" event={"ID":"5710c05e-19b5-486d-8697-1cde26e40a86","Type":"ContainerStarted","Data":"cd067b21c8f0e5d3259bc06b52c629ecf3940d707eca135010206a425d38d4bb"} Sep 05 06:27:22.611460 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.611431 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-g8dlg"] Sep 05 06:27:22.615571 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.615553 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:22.618002 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.617982 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 05 06:27:22.618806 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.618789 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 05 06:27:22.618864 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.618806 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 05 06:27:22.618864 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.618811 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 05 06:27:22.618957 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.618878 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-cfr9x\"" Sep 05 06:27:22.622365 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.622336 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-g8dlg"] Sep 05 06:27:22.634112 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.634095 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m42bc_302f2dc8-13ec-46b9-b694-f66048d4e2c3/dns-node-resolver/0.log" Sep 05 06:27:22.661572 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.661548 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/19a51967-4894-4256-a709-4d1d853b6892-signing-cabundle\") pod \"service-ca-75d99c484-g8dlg\" (UID: \"19a51967-4894-4256-a709-4d1d853b6892\") " pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:22.661672 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.661584 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/19a51967-4894-4256-a709-4d1d853b6892-signing-key\") pod \"service-ca-75d99c484-g8dlg\" (UID: \"19a51967-4894-4256-a709-4d1d853b6892\") " pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:22.661672 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.661602 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sxb7\" (UniqueName: \"kubernetes.io/projected/19a51967-4894-4256-a709-4d1d853b6892-kube-api-access-4sxb7\") pod \"service-ca-75d99c484-g8dlg\" (UID: \"19a51967-4894-4256-a709-4d1d853b6892\") " pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:22.762906 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.762846 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/19a51967-4894-4256-a709-4d1d853b6892-signing-cabundle\") pod \"service-ca-75d99c484-g8dlg\" (UID: \"19a51967-4894-4256-a709-4d1d853b6892\") " pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:22.762906 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.762900 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/19a51967-4894-4256-a709-4d1d853b6892-signing-key\") pod \"service-ca-75d99c484-g8dlg\" (UID: \"19a51967-4894-4256-a709-4d1d853b6892\") " pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:22.763063 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.762921 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sxb7\" (UniqueName: \"kubernetes.io/projected/19a51967-4894-4256-a709-4d1d853b6892-kube-api-access-4sxb7\") pod \"service-ca-75d99c484-g8dlg\" (UID: \"19a51967-4894-4256-a709-4d1d853b6892\") " pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:22.763587 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.763562 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/19a51967-4894-4256-a709-4d1d853b6892-signing-cabundle\") pod \"service-ca-75d99c484-g8dlg\" (UID: \"19a51967-4894-4256-a709-4d1d853b6892\") " pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:22.765662 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.765635 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/19a51967-4894-4256-a709-4d1d853b6892-signing-key\") pod \"service-ca-75d99c484-g8dlg\" (UID: \"19a51967-4894-4256-a709-4d1d853b6892\") " pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:22.771741 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.771715 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sxb7\" (UniqueName: \"kubernetes.io/projected/19a51967-4894-4256-a709-4d1d853b6892-kube-api-access-4sxb7\") pod \"service-ca-75d99c484-g8dlg\" (UID: \"19a51967-4894-4256-a709-4d1d853b6892\") " pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:22.924260 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:22.924190 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-g8dlg" Sep 05 06:27:23.197658 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:23.197593 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-g8dlg"] Sep 05 06:27:23.268091 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:23.268056 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:23.268255 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:23.268203 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 05 06:27:23.268322 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:23.268270 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls podName:2c15fd53-f580-45ab-aacf-5a21e7500817 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:31.268254801 +0000 UTC m=+147.951759686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6r7ml" (UID: "2c15fd53-f580-45ab-aacf-5a21e7500817") : secret "cluster-monitoring-operator-tls" not found Sep 05 06:27:23.269255 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:23.269227 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a51967_4894_4256_a709_4d1d853b6892.slice/crio-871313cfb4907a70a22dd2fd39336b538283455f3578731aeff3c67939390db0 WatchSource:0}: Error finding container 871313cfb4907a70a22dd2fd39336b538283455f3578731aeff3c67939390db0: Status 404 returned error can't find the container with id 871313cfb4907a70a22dd2fd39336b538283455f3578731aeff3c67939390db0 Sep 05 06:27:23.317193 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:23.317163 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-g8dlg" event={"ID":"19a51967-4894-4256-a709-4d1d853b6892","Type":"ContainerStarted","Data":"871313cfb4907a70a22dd2fd39336b538283455f3578731aeff3c67939390db0"} Sep 05 06:27:23.430249 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:23.430226 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4bf69_f64eb380-73df-46a4-9766-dd8d99bc4625/node-ca/0.log" Sep 05 06:27:24.174554 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:24.174514 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:24.174554 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:24.174559 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:24.174757 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:24.174630 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 05 06:27:24.174757 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:24.174664 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 05 06:27:24.174757 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:24.174680 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm podName:49f311ef-2d4d-4cf1-93e4-4138b14525a5 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:28.174666552 +0000 UTC m=+144.858171437 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b29nx" (UID: "49f311ef-2d4d-4cf1-93e4-4138b14525a5") : configmap "kube-rbac-proxy" not found Sep 05 06:27:24.174757 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:24.174717 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls podName:49f311ef-2d4d-4cf1-93e4-4138b14525a5 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:28.17470484 +0000 UTC m=+144.858209724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b29nx" (UID: "49f311ef-2d4d-4cf1-93e4-4138b14525a5") : secret "insights-runtime-extractor-tls" not found Sep 05 06:27:24.231734 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:24.231707 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hwz4f_2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4/serve-healthcheck-canary/0.log" Sep 05 06:27:24.324089 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:24.324041 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb" event={"ID":"5710c05e-19b5-486d-8697-1cde26e40a86","Type":"ContainerStarted","Data":"f6215179a5a64f8bbcadc3b9f98dc1b7903253dca7c5c9afe2ec13904243367a"} Sep 05 06:27:24.324543 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:24.324132 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb" event={"ID":"5710c05e-19b5-486d-8697-1cde26e40a86","Type":"ContainerStarted","Data":"1d40fd67cbc065063871d052e383600f045ef59e5030755ca540cad9580170bf"} Sep 05 06:27:24.326546 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:24.326503 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-g8dlg" event={"ID":"19a51967-4894-4256-a709-4d1d853b6892","Type":"ContainerStarted","Data":"17b0440dc51717893cb05f3ef03a39155e00e78c9575cd85770d01053c4aca00"} Sep 05 06:27:24.347128 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:24.347071 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-4f2jb" podStartSLOduration=2.267798041 podStartE2EDuration="3.347055757s" totalImagesPullingTime="1.079257716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:24.346283367 +0000 UTC m=+141.029788275" watchObservedRunningTime="2026-09-05 06:27:24.347055757 +0000 UTC m=+141.030560664" Sep 05 06:27:24.365142 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:24.365092 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-g8dlg" podStartSLOduration=2.365075377 podStartE2EDuration="2.365075377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:24.364536681 +0000 UTC m=+141.048041589" watchObservedRunningTime="2026-09-05 06:27:24.365075377 +0000 UTC m=+141.048580337" Sep 05 06:27:26.074203 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:26.074172 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:26.074613 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:26.074231 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:26.074613 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:26.074540 2583 scope.go:117] "RemoveContainer" containerID="1012eba5de6ab6e8cbe316a4fdb36a1ea5dd24f137d09148090cbd82331206d9" Sep 05 06:27:26.074713 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:26.074675 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-8bx7z_openshift-console-operator(a6b9ff6f-b0f0-43eb-8516-1939b0a95609)\"" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" podUID="a6b9ff6f-b0f0-43eb-8516-1939b0a95609" Sep 05 06:27:26.333252 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:26.333184 2583 scope.go:117] "RemoveContainer" containerID="1012eba5de6ab6e8cbe316a4fdb36a1ea5dd24f137d09148090cbd82331206d9" Sep 05 06:27:26.333373 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:26.333322 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-8bx7z_openshift-console-operator(a6b9ff6f-b0f0-43eb-8516-1939b0a95609)\"" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" podUID="a6b9ff6f-b0f0-43eb-8516-1939b0a95609" Sep 05 06:27:28.203566 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:28.203518 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:28.203962 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:28.203616 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:28.203962 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:28.203647 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 05 06:27:28.203962 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:28.203709 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 05 06:27:28.203962 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:28.203725 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm podName:49f311ef-2d4d-4cf1-93e4-4138b14525a5 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:36.203709014 +0000 UTC m=+152.887213899 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-b29nx" (UID: "49f311ef-2d4d-4cf1-93e4-4138b14525a5") : configmap "kube-rbac-proxy" not found Sep 05 06:27:28.203962 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:28.203751 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls podName:49f311ef-2d4d-4cf1-93e4-4138b14525a5 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:36.203740406 +0000 UTC m=+152.887245291 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-b29nx" (UID: "49f311ef-2d4d-4cf1-93e4-4138b14525a5") : secret "insights-runtime-extractor-tls" not found Sep 05 06:27:31.325939 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:31.325901 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:31.328223 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:31.328190 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2c15fd53-f580-45ab-aacf-5a21e7500817-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6r7ml\" (UID: \"2c15fd53-f580-45ab-aacf-5a21e7500817\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:31.452172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:31.452136 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" Sep 05 06:27:31.722722 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:31.722647 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml"] Sep 05 06:27:31.725873 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:31.725834 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c15fd53_f580_45ab_aacf_5a21e7500817.slice/crio-1c7a7ba76b1fbc4dbe1804e70365123cd667861e1e44c87fd6844c93f880e72c WatchSource:0}: Error finding container 1c7a7ba76b1fbc4dbe1804e70365123cd667861e1e44c87fd6844c93f880e72c: Status 404 returned error can't find the container with id 1c7a7ba76b1fbc4dbe1804e70365123cd667861e1e44c87fd6844c93f880e72c Sep 05 06:27:32.351666 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:32.351624 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" event={"ID":"2c15fd53-f580-45ab-aacf-5a21e7500817","Type":"ContainerStarted","Data":"1c7a7ba76b1fbc4dbe1804e70365123cd667861e1e44c87fd6844c93f880e72c"} Sep 05 06:27:34.361294 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:34.361256 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" event={"ID":"2c15fd53-f580-45ab-aacf-5a21e7500817","Type":"ContainerStarted","Data":"b42fdf2dcac6b1f3964c021d21a08d18f7bc498806e9ddc7b6ffb6d4dd65231c"} Sep 05 06:27:34.381613 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:34.381569 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6r7ml" podStartSLOduration=17.679130789 podStartE2EDuration="19.381555236s" totalImagesPullingTime="1.702424447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:34.380256565 +0000 UTC m=+151.063761472" watchObservedRunningTime="2026-09-05 06:27:34.381555236 +0000 UTC m=+151.065060142" Sep 05 06:27:36.268152 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:36.268117 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:36.268152 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:36.268159 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:36.268647 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:36.268630 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49f311ef-2d4d-4cf1-93e4-4138b14525a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:36.270407 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:36.270381 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49f311ef-2d4d-4cf1-93e4-4138b14525a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b29nx\" (UID: \"49f311ef-2d4d-4cf1-93e4-4138b14525a5\") " pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:36.390605 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:36.390564 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b29nx" Sep 05 06:27:36.659548 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:36.659443 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b29nx"] Sep 05 06:27:36.663387 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:36.663364 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f311ef_2d4d_4cf1_93e4_4138b14525a5.slice/crio-56cb6bc46fbe84d8f17293287f3533f0d76519847a1800aa8274c2318a0b5a0f WatchSource:0}: Error finding container 56cb6bc46fbe84d8f17293287f3533f0d76519847a1800aa8274c2318a0b5a0f: Status 404 returned error can't find the container with id 56cb6bc46fbe84d8f17293287f3533f0d76519847a1800aa8274c2318a0b5a0f Sep 05 06:27:36.816561 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:36.816533 2583 scope.go:117] "RemoveContainer" containerID="1012eba5de6ab6e8cbe316a4fdb36a1ea5dd24f137d09148090cbd82331206d9" Sep 05 06:27:37.373470 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:37.373448 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:27:37.373875 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:37.373858 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/1.log" Sep 05 06:27:37.373939 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:37.373896 2583 generic.go:356] "Generic (PLEG): container finished" podID="a6b9ff6f-b0f0-43eb-8516-1939b0a95609" containerID="a5c0a4ba504f91c170b75896e6f9459e888c80b89c9c019041a6c823c4f9223c" exitCode=255 Sep 05 06:27:37.373994 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:37.373972 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" event={"ID":"a6b9ff6f-b0f0-43eb-8516-1939b0a95609","Type":"ContainerDied","Data":"a5c0a4ba504f91c170b75896e6f9459e888c80b89c9c019041a6c823c4f9223c"} Sep 05 06:27:37.374043 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:37.374026 2583 scope.go:117] "RemoveContainer" containerID="1012eba5de6ab6e8cbe316a4fdb36a1ea5dd24f137d09148090cbd82331206d9" Sep 05 06:27:37.374322 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:37.374306 2583 scope.go:117] "RemoveContainer" containerID="a5c0a4ba504f91c170b75896e6f9459e888c80b89c9c019041a6c823c4f9223c" Sep 05 06:27:37.374510 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:37.374489 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-8bx7z_openshift-console-operator(a6b9ff6f-b0f0-43eb-8516-1939b0a95609)\"" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" podUID="a6b9ff6f-b0f0-43eb-8516-1939b0a95609" Sep 05 06:27:37.375433 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:37.375416 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b29nx" event={"ID":"49f311ef-2d4d-4cf1-93e4-4138b14525a5","Type":"ContainerStarted","Data":"3f95aa5c6132a2ad780e909fdd070748d16c390d2b9848993d5fd73ae9c3ead2"} Sep 05 06:27:37.375518 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:37.375455 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b29nx" event={"ID":"49f311ef-2d4d-4cf1-93e4-4138b14525a5","Type":"ContainerStarted","Data":"56cb6bc46fbe84d8f17293287f3533f0d76519847a1800aa8274c2318a0b5a0f"} Sep 05 06:27:38.380748 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:38.380714 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b29nx" event={"ID":"49f311ef-2d4d-4cf1-93e4-4138b14525a5","Type":"ContainerStarted","Data":"335fc365f4e1aa5ddd4c45268dbb299b0ff1771a0d7e01325e658aff710e9af0"} Sep 05 06:27:38.382296 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:38.382276 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:27:39.390634 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:39.390598 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b29nx" event={"ID":"49f311ef-2d4d-4cf1-93e4-4138b14525a5","Type":"ContainerStarted","Data":"2b05a446c9646418e49a9c1f6cdf1a2be5131f916ef848aaa693aec3535c5bf5"} Sep 05 06:27:39.409709 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:39.409661 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-b29nx" podStartSLOduration=17.417815673 podStartE2EDuration="19.409647334s" totalImagesPullingTime="1.991831661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:27:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:39.408384432 +0000 UTC m=+156.091889338" watchObservedRunningTime="2026-09-05 06:27:39.409647334 +0000 UTC m=+156.093152241" Sep 05 06:27:41.148087 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:41.148048 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-p8fq5" podUID="a3fff948-476a-4e52-824c-62d88e038d9d" Sep 05 06:27:41.396454 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:41.396428 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p8fq5" Sep 05 06:27:41.826003 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:41.825968 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-m8fn4" podUID="68080680-0551-487c-80ed-b93acb2f68f8" Sep 05 06:27:44.607038 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.606983 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-fwvvg"] Sep 05 06:27:44.610112 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.610092 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" Sep 05 06:27:44.614397 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.613656 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 05 06:27:44.614397 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.613992 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qm2hp\"" Sep 05 06:27:44.614397 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.614232 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 05 06:27:44.618557 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.618437 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-z9gtc"] Sep 05 06:27:44.621469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.621450 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-fwvvg"] Sep 05 06:27:44.621585 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.621563 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-z9gtc" Sep 05 06:27:44.624555 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.624528 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-j568m\"" Sep 05 06:27:44.631578 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.631555 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-z9gtc"] Sep 05 06:27:44.690421 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.690387 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h"] Sep 05 06:27:44.693716 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.693698 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h" Sep 05 06:27:44.696041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.696023 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 05 06:27:44.696171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.696059 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-87d6w\"" Sep 05 06:27:44.702060 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.702038 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h"] Sep 05 06:27:44.720589 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.720562 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d115bb76-6396-40b9-b87a-1e1ab8d31282-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fwvvg\" (UID: \"d115bb76-6396-40b9-b87a-1e1ab8d31282\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" Sep 05 06:27:44.720783 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.720596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mzzp\" (UniqueName: \"kubernetes.io/projected/94eca2d2-249d-4e8b-9375-66b9313b958e-kube-api-access-2mzzp\") pod \"network-check-source-685456655f-z9gtc\" (UID: \"94eca2d2-249d-4e8b-9375-66b9313b958e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-z9gtc" Sep 05 06:27:44.720783 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.720619 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d115bb76-6396-40b9-b87a-1e1ab8d31282-nginx-conf\") pod \"networking-console-plugin-79977f9844-fwvvg\" (UID: \"d115bb76-6396-40b9-b87a-1e1ab8d31282\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" Sep 05 06:27:44.821278 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.821251 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d115bb76-6396-40b9-b87a-1e1ab8d31282-nginx-conf\") pod \"networking-console-plugin-79977f9844-fwvvg\" (UID: \"d115bb76-6396-40b9-b87a-1e1ab8d31282\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" Sep 05 06:27:44.821456 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.821316 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/32e578ed-c724-41ac-9734-d82e91cfefb2-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-5cq9h\" (UID: \"32e578ed-c724-41ac-9734-d82e91cfefb2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h" Sep 05 06:27:44.821456 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.821344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d115bb76-6396-40b9-b87a-1e1ab8d31282-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fwvvg\" (UID: \"d115bb76-6396-40b9-b87a-1e1ab8d31282\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" Sep 05 06:27:44.821456 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.821361 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mzzp\" (UniqueName: \"kubernetes.io/projected/94eca2d2-249d-4e8b-9375-66b9313b958e-kube-api-access-2mzzp\") pod \"network-check-source-685456655f-z9gtc\" (UID: \"94eca2d2-249d-4e8b-9375-66b9313b958e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-z9gtc" Sep 05 06:27:44.822019 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.821990 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d115bb76-6396-40b9-b87a-1e1ab8d31282-nginx-conf\") pod \"networking-console-plugin-79977f9844-fwvvg\" (UID: \"d115bb76-6396-40b9-b87a-1e1ab8d31282\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" Sep 05 06:27:44.823718 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.823699 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d115bb76-6396-40b9-b87a-1e1ab8d31282-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fwvvg\" (UID: \"d115bb76-6396-40b9-b87a-1e1ab8d31282\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" Sep 05 06:27:44.832616 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.832590 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mzzp\" (UniqueName: \"kubernetes.io/projected/94eca2d2-249d-4e8b-9375-66b9313b958e-kube-api-access-2mzzp\") pod \"network-check-source-685456655f-z9gtc\" (UID: \"94eca2d2-249d-4e8b-9375-66b9313b958e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-z9gtc" Sep 05 06:27:44.922451 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.922371 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" Sep 05 06:27:44.922662 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.922605 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/32e578ed-c724-41ac-9734-d82e91cfefb2-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-5cq9h\" (UID: \"32e578ed-c724-41ac-9734-d82e91cfefb2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h" Sep 05 06:27:44.925296 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.925262 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/32e578ed-c724-41ac-9734-d82e91cfefb2-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-5cq9h\" (UID: \"32e578ed-c724-41ac-9734-d82e91cfefb2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h" Sep 05 06:27:44.929606 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:44.929586 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-z9gtc" Sep 05 06:27:45.003021 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:45.002992 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h" Sep 05 06:27:45.192418 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:45.192392 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-fwvvg"] Sep 05 06:27:45.194606 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:45.194581 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd115bb76_6396_40b9_b87a_1e1ab8d31282.slice/crio-4d4d538f392771c04bd3898681ddcfa2115743a69a7fa367c0fedc42801cdac5 WatchSource:0}: Error finding container 4d4d538f392771c04bd3898681ddcfa2115743a69a7fa367c0fedc42801cdac5: Status 404 returned error can't find the container with id 4d4d538f392771c04bd3898681ddcfa2115743a69a7fa367c0fedc42801cdac5 Sep 05 06:27:45.215098 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:45.215076 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-z9gtc"] Sep 05 06:27:45.217821 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:45.217797 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94eca2d2_249d_4e8b_9375_66b9313b958e.slice/crio-acfc5bc96c152fc5ade3a91095dfd0b1a0f192d6e0fc375610ee9f4ef49797bb WatchSource:0}: Error finding container acfc5bc96c152fc5ade3a91095dfd0b1a0f192d6e0fc375610ee9f4ef49797bb: Status 404 returned error can't find the container with id acfc5bc96c152fc5ade3a91095dfd0b1a0f192d6e0fc375610ee9f4ef49797bb Sep 05 06:27:45.271865 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:45.271839 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h"] Sep 05 06:27:45.274357 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:45.274333 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e578ed_c724_41ac_9734_d82e91cfefb2.slice/crio-c4ff2ec1ba22d1d23465c7d8d322515f5f0b22ba62f9c5a5a162f3f4615b7d26 WatchSource:0}: Error finding container c4ff2ec1ba22d1d23465c7d8d322515f5f0b22ba62f9c5a5a162f3f4615b7d26: Status 404 returned error can't find the container with id c4ff2ec1ba22d1d23465c7d8d322515f5f0b22ba62f9c5a5a162f3f4615b7d26 Sep 05 06:27:45.411419 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:45.411381 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-z9gtc" event={"ID":"94eca2d2-249d-4e8b-9375-66b9313b958e","Type":"ContainerStarted","Data":"2a3d181c1206f31e5b867a85678458941ce81a5e8596a24862e4aeee7f033810"} Sep 05 06:27:45.411644 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:45.411434 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-z9gtc" event={"ID":"94eca2d2-249d-4e8b-9375-66b9313b958e","Type":"ContainerStarted","Data":"acfc5bc96c152fc5ade3a91095dfd0b1a0f192d6e0fc375610ee9f4ef49797bb"} Sep 05 06:27:45.412730 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:45.412697 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h" event={"ID":"32e578ed-c724-41ac-9734-d82e91cfefb2","Type":"ContainerStarted","Data":"c4ff2ec1ba22d1d23465c7d8d322515f5f0b22ba62f9c5a5a162f3f4615b7d26"} Sep 05 06:27:45.413891 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:45.413865 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" event={"ID":"d115bb76-6396-40b9-b87a-1e1ab8d31282","Type":"ContainerStarted","Data":"4d4d538f392771c04bd3898681ddcfa2115743a69a7fa367c0fedc42801cdac5"} Sep 05 06:27:45.428614 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:45.428566 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-z9gtc" podStartSLOduration=1.428546827 podStartE2EDuration="1.428546827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:45.426696123 +0000 UTC m=+162.110201031" watchObservedRunningTime="2026-09-05 06:27:45.428546827 +0000 UTC m=+162.112051737" Sep 05 06:27:46.030886 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:46.030846 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:27:46.033276 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:46.033251 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fff948-476a-4e52-824c-62d88e038d9d-metrics-tls\") pod \"dns-default-p8fq5\" (UID: \"a3fff948-476a-4e52-824c-62d88e038d9d\") " pod="openshift-dns/dns-default-p8fq5" Sep 05 06:27:46.074595 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:46.074558 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:46.074767 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:46.074668 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:46.074991 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:46.074977 2583 scope.go:117] "RemoveContainer" containerID="a5c0a4ba504f91c170b75896e6f9459e888c80b89c9c019041a6c823c4f9223c" Sep 05 06:27:46.075175 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:46.075153 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-8bx7z_openshift-console-operator(a6b9ff6f-b0f0-43eb-8516-1939b0a95609)\"" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" podUID="a6b9ff6f-b0f0-43eb-8516-1939b0a95609" Sep 05 06:27:46.200202 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:46.200171 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bxnqj\"" Sep 05 06:27:46.208231 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:46.208201 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p8fq5" Sep 05 06:27:46.417533 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:46.417444 2583 scope.go:117] "RemoveContainer" containerID="a5c0a4ba504f91c170b75896e6f9459e888c80b89c9c019041a6c823c4f9223c" Sep 05 06:27:46.417682 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:46.417601 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-8bx7z_openshift-console-operator(a6b9ff6f-b0f0-43eb-8516-1939b0a95609)\"" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" podUID="a6b9ff6f-b0f0-43eb-8516-1939b0a95609" Sep 05 06:27:46.863982 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:46.863950 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p8fq5"] Sep 05 06:27:46.867965 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:46.867928 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3fff948_476a_4e52_824c_62d88e038d9d.slice/crio-c68a8c05dc4398b72e6cba7a2b5654cf4c27e0dc47965b84d56df12d3755696b WatchSource:0}: Error finding container c68a8c05dc4398b72e6cba7a2b5654cf4c27e0dc47965b84d56df12d3755696b: Status 404 returned error can't find the container with id c68a8c05dc4398b72e6cba7a2b5654cf4c27e0dc47965b84d56df12d3755696b Sep 05 06:27:47.422510 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:47.422457 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p8fq5" event={"ID":"a3fff948-476a-4e52-824c-62d88e038d9d","Type":"ContainerStarted","Data":"c68a8c05dc4398b72e6cba7a2b5654cf4c27e0dc47965b84d56df12d3755696b"} Sep 05 06:27:47.424300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:47.424269 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h" event={"ID":"32e578ed-c724-41ac-9734-d82e91cfefb2","Type":"ContainerStarted","Data":"f63d064326307e23d5bcd2bfb705a14be552decfa162b252a26951e34b16a5f3"} Sep 05 06:27:47.424436 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:47.424392 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h" Sep 05 06:27:47.426051 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:47.426029 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" event={"ID":"d115bb76-6396-40b9-b87a-1e1ab8d31282","Type":"ContainerStarted","Data":"d4114be60298ccc0d1676a3075c658049b9dc26a2dabd66d5984cc42193885db"} Sep 05 06:27:47.430115 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:47.430093 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h" Sep 05 06:27:47.442152 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:47.442107 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5cq9h" podStartSLOduration=2.116594978 podStartE2EDuration="3.442090622s" totalImagesPullingTime="1.325495644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:47.441895294 +0000 UTC m=+164.125400202" watchObservedRunningTime="2026-09-05 06:27:47.442090622 +0000 UTC m=+164.125595532" Sep 05 06:27:47.459427 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:47.459336 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-fwvvg" podStartSLOduration=2.059972322 podStartE2EDuration="3.459319753s" totalImagesPullingTime="1.399347431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:47.458611958 +0000 UTC m=+164.142116866" watchObservedRunningTime="2026-09-05 06:27:47.459319753 +0000 UTC m=+164.142824662" Sep 05 06:27:48.057300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.057266 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-fwzxk"] Sep 05 06:27:48.060886 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.060866 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.063659 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.063635 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 05 06:27:48.064058 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.064034 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 05 06:27:48.064058 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.064051 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tdqwp\"" Sep 05 06:27:48.064711 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.064692 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 05 06:27:48.074247 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.074226 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-fwzxk"] Sep 05 06:27:48.149657 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.149620 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.149798 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.149667 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.149798 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.149750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.149798 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.149790 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msld4\" (UniqueName: \"kubernetes.io/projected/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-kube-api-access-msld4\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.250281 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.250239 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.250281 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.250292 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.250604 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.250337 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.250604 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.250368 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msld4\" (UniqueName: \"kubernetes.io/projected/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-kube-api-access-msld4\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.251137 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.251113 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.253023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.253000 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.253432 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.253414 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.263142 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.263111 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msld4\" (UniqueName: \"kubernetes.io/projected/a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451-kube-api-access-msld4\") pod \"prometheus-operator-6f4946db56-fwzxk\" (UID: \"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.372471 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.372379 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" Sep 05 06:27:48.678949 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:48.678917 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-fwzxk"] Sep 05 06:27:48.681887 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:48.681863 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3bb8608_d9aa_4bbc_8e42_8e22dd3e0451.slice/crio-66c6b26bade760d2ccbec9f9cff155291580dfddfbefd841a264157b5c3d176e WatchSource:0}: Error finding container 66c6b26bade760d2ccbec9f9cff155291580dfddfbefd841a264157b5c3d176e: Status 404 returned error can't find the container with id 66c6b26bade760d2ccbec9f9cff155291580dfddfbefd841a264157b5c3d176e Sep 05 06:27:49.435131 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:49.435096 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p8fq5" event={"ID":"a3fff948-476a-4e52-824c-62d88e038d9d","Type":"ContainerStarted","Data":"65d7fd8ef02e5a15b6e03443f98b7d349b4874a46209220d6c0f85011d010b2c"} Sep 05 06:27:49.435319 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:49.435158 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p8fq5" event={"ID":"a3fff948-476a-4e52-824c-62d88e038d9d","Type":"ContainerStarted","Data":"05c96f6e97b829fca54b5a47dd0edf31bc8ea5c6c5efb02c8d34a7af8d0a0a81"} Sep 05 06:27:49.435319 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:49.435193 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p8fq5" Sep 05 06:27:49.436545 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:49.436512 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" event={"ID":"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451","Type":"ContainerStarted","Data":"66c6b26bade760d2ccbec9f9cff155291580dfddfbefd841a264157b5c3d176e"} Sep 05 06:27:49.453748 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:49.453698 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-p8fq5" podStartSLOduration=129.860390866 podStartE2EDuration="2m11.453678365s" totalImagesPullingTime="1.593287499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:49.451712639 +0000 UTC m=+166.135217547" watchObservedRunningTime="2026-09-05 06:27:49.453678365 +0000 UTC m=+166.137183273" Sep 05 06:27:50.441945 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:50.441907 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" event={"ID":"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451","Type":"ContainerStarted","Data":"5b1d47f8a09625e2cbc7129d4f40518e7d7bffcbe6ffba4c0e596a5bb6eca88b"} Sep 05 06:27:50.442291 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:50.441971 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" event={"ID":"a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451","Type":"ContainerStarted","Data":"8a5db46336a2415661a22f7cdd810e734e211864afe127d0849b788da56f7f5d"} Sep 05 06:27:50.462195 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:50.462149 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-fwzxk" podStartSLOduration=1.278368837 podStartE2EDuration="2.462135566s" totalImagesPullingTime="1.183766729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:50.461130018 +0000 UTC m=+167.144634937" watchObservedRunningTime="2026-09-05 06:27:50.462135566 +0000 UTC m=+167.145640474" Sep 05 06:27:52.509430 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.509392 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-r25d6"] Sep 05 06:27:52.543595 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.543557 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-r25d6"] Sep 05 06:27:52.543734 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.543686 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.546667 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.546643 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 05 06:27:52.546906 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.546886 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 05 06:27:52.550649 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.550279 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 05 06:27:52.550649 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.550507 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-6nkwt\"" Sep 05 06:27:52.550949 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.550931 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qnd7c"] Sep 05 06:27:52.567543 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.567524 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.572376 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.572219 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 05 06:27:52.572470 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.572393 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 05 06:27:52.572853 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.572829 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sgg5g\"" Sep 05 06:27:52.573226 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.573210 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 05 06:27:52.684843 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.684802 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/276312ef-5e4f-4a22-ae1c-600a2885e110-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.684843 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.684844 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e3bf338-be1e-4459-ac3e-a372a1696832-metrics-client-ca\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.685041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.684867 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/276312ef-5e4f-4a22-ae1c-600a2885e110-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.685041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.684882 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wr98\" (UniqueName: \"kubernetes.io/projected/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-api-access-6wr98\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.685041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.684932 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e3bf338-be1e-4459-ac3e-a372a1696832-root\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.685041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.684971 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-textfile\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.685041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.684999 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbvqh\" (UniqueName: \"kubernetes.io/projected/0e3bf338-be1e-4459-ac3e-a372a1696832-kube-api-access-zbvqh\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.685041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.685029 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.685226 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.685085 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.685226 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.685112 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-tls\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.685226 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.685156 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.685226 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.685182 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e3bf338-be1e-4459-ac3e-a372a1696832-sys\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.685226 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.685205 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-wtmp\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.685374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.685241 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.685374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.685281 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-accelerators-collector-config\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.785877 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.785794 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.785877 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.785840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-accelerators-collector-config\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.785903 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/276312ef-5e4f-4a22-ae1c-600a2885e110-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.786092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.785926 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e3bf338-be1e-4459-ac3e-a372a1696832-metrics-client-ca\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.785953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/276312ef-5e4f-4a22-ae1c-600a2885e110-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.786092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.785974 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wr98\" (UniqueName: \"kubernetes.io/projected/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-api-access-6wr98\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.786092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.785998 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e3bf338-be1e-4459-ac3e-a372a1696832-root\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786019 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-textfile\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786044 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbvqh\" (UniqueName: \"kubernetes.io/projected/0e3bf338-be1e-4459-ac3e-a372a1696832-kube-api-access-zbvqh\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786074 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.786494 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.786494 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786134 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-tls\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786494 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786169 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786494 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786197 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e3bf338-be1e-4459-ac3e-a372a1696832-sys\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786494 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786219 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-wtmp\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786494 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786388 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-wtmp\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786494 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786441 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e3bf338-be1e-4459-ac3e-a372a1696832-root\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786933 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786543 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-accelerators-collector-config\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786933 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:52.786556 2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 05 06:27:52.786933 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:52.786613 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-tls podName:0e3bf338-be1e-4459-ac3e-a372a1696832 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:53.286595279 +0000 UTC m=+169.970100178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-tls") pod "node-exporter-qnd7c" (UID: "0e3bf338-be1e-4459-ac3e-a372a1696832") : secret "node-exporter-tls" not found Sep 05 06:27:52.786933 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786625 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.786933 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786655 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/276312ef-5e4f-4a22-ae1c-600a2885e110-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.786933 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786824 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-textfile\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.786933 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.786839 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e3bf338-be1e-4459-ac3e-a372a1696832-sys\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.787290 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.787222 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/276312ef-5e4f-4a22-ae1c-600a2885e110-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.787364 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.787340 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e3bf338-be1e-4459-ac3e-a372a1696832-metrics-client-ca\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.788896 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.788872 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.789248 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.789225 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.789426 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.789408 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.797353 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.797330 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wr98\" (UniqueName: \"kubernetes.io/projected/276312ef-5e4f-4a22-ae1c-600a2885e110-kube-api-access-6wr98\") pod \"kube-state-metrics-74cb57df7-r25d6\" (UID: \"276312ef-5e4f-4a22-ae1c-600a2885e110\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:52.797877 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.797849 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbvqh\" (UniqueName: \"kubernetes.io/projected/0e3bf338-be1e-4459-ac3e-a372a1696832-kube-api-access-zbvqh\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:52.854562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:52.854532 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" Sep 05 06:27:53.160858 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.160775 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-r25d6"] Sep 05 06:27:53.163286 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:53.163261 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod276312ef_5e4f_4a22_ae1c_600a2885e110.slice/crio-ef986dd70c3b5043cb0cfcbba99e74244d7daa53d4510d74b912ec478027d394 WatchSource:0}: Error finding container ef986dd70c3b5043cb0cfcbba99e74244d7daa53d4510d74b912ec478027d394: Status 404 returned error can't find the container with id ef986dd70c3b5043cb0cfcbba99e74244d7daa53d4510d74b912ec478027d394 Sep 05 06:27:53.290535 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.290508 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-tls\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:53.292667 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.292642 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e3bf338-be1e-4459-ac3e-a372a1696832-node-exporter-tls\") pod \"node-exporter-qnd7c\" (UID: \"0e3bf338-be1e-4459-ac3e-a372a1696832\") " pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:53.456392 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.456314 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" event={"ID":"276312ef-5e4f-4a22-ae1c-600a2885e110","Type":"ContainerStarted","Data":"ef986dd70c3b5043cb0cfcbba99e74244d7daa53d4510d74b912ec478027d394"} Sep 05 06:27:53.476573 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.476548 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qnd7c" Sep 05 06:27:53.497866 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.497838 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 05 06:27:53.514922 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.514902 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.517927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.517636 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 05 06:27:53.517927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.517658 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 05 06:27:53.517927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.517681 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 05 06:27:53.517927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.517741 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 05 06:27:53.517927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.517636 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 05 06:27:53.517927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.517919 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 05 06:27:53.518391 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.518065 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 05 06:27:53.518391 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.518160 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 05 06:27:53.518391 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.518255 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qbfr2\"" Sep 05 06:27:53.518573 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.518539 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 05 06:27:53.518999 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:53.518972 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3bf338_be1e_4459_ac3e_a372a1696832.slice/crio-5067b6fcca685f7105e59622776cbc61861daf2dd048cf3a23ea51ec1d2c07ae WatchSource:0}: Error finding container 5067b6fcca685f7105e59622776cbc61861daf2dd048cf3a23ea51ec1d2c07ae: Status 404 returned error can't find the container with id 5067b6fcca685f7105e59622776cbc61861daf2dd048cf3a23ea51ec1d2c07ae Sep 05 06:27:53.520880 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.520662 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 05 06:27:53.694233 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694186 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-volume\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694413 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694241 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-out\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694413 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694281 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-web-config\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694413 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694325 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694413 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694357 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694413 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694388 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694447 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694511 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694538 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694601 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694635 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62988\" (UniqueName: \"kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-kube-api-access-62988\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694679 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.694963 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.694754 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.795787 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.795748 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-volume\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.795955 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.795808 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-out\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.795955 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.795832 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-web-config\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.795955 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.795870 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.795955 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.795901 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.796240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.796210 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.796387 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.796370 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.796521 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.796504 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.796649 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.796633 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.796754 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.796740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.796857 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.796842 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62988\" (UniqueName: \"kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-kube-api-access-62988\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.797386 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.796934 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.797386 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.796947 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.797386 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.797042 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.797682 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.797619 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.798991 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:53.798500 2583 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 05 06:27:53.798991 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:27:53.798564 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-main-tls podName:689e8836-47ec-4bb4-9d1a-90c0c43c48bc nodeName:}" failed. No retries permitted until 2026-09-05 06:27:54.298545856 +0000 UTC m=+170.982050746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc") : secret "alertmanager-main-tls" not found Sep 05 06:27:53.798991 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.798869 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.801144 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.800662 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-out\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.801144 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.800737 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.801294 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.801149 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.802641 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.802093 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.802641 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.802532 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.802867 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.802844 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-volume\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.803190 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.802966 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.804515 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.804469 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-web-config\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.815581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.810934 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62988\" (UniqueName: \"kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-kube-api-access-62988\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:53.818443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:53.818419 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:27:54.302271 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:54.302239 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:54.304886 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:54.304863 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:54.428417 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:54.428376 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:27:54.462527 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:54.462494 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnd7c" event={"ID":"0e3bf338-be1e-4459-ac3e-a372a1696832","Type":"ContainerStarted","Data":"5067b6fcca685f7105e59622776cbc61861daf2dd048cf3a23ea51ec1d2c07ae"} Sep 05 06:27:55.023065 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.023039 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 05 06:27:55.025078 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:55.025049 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod689e8836_47ec_4bb4_9d1a_90c0c43c48bc.slice/crio-5054cd8b70d4f474a20f92b7d7022dee1ba2ba6c2e8f564bb943fdc82df8bdf0 WatchSource:0}: Error finding container 5054cd8b70d4f474a20f92b7d7022dee1ba2ba6c2e8f564bb943fdc82df8bdf0: Status 404 returned error can't find the container with id 5054cd8b70d4f474a20f92b7d7022dee1ba2ba6c2e8f564bb943fdc82df8bdf0 Sep 05 06:27:55.415853 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.415679 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7bff8865f5-6glhx"] Sep 05 06:27:55.420962 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.420885 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.423852 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.423827 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 05 06:27:55.424113 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.423831 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-45r8x\"" Sep 05 06:27:55.424300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.424277 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 05 06:27:55.424701 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.424549 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 05 06:27:55.424701 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.424568 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-53lsdusip385l\"" Sep 05 06:27:55.424701 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.424562 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 05 06:27:55.424913 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.424801 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 05 06:27:55.434603 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.434580 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7bff8865f5-6glhx"] Sep 05 06:27:55.468496 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.468452 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerStarted","Data":"5054cd8b70d4f474a20f92b7d7022dee1ba2ba6c2e8f564bb943fdc82df8bdf0"} Sep 05 06:27:55.470497 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.470441 2583 generic.go:356] "Generic (PLEG): container finished" podID="0e3bf338-be1e-4459-ac3e-a372a1696832" containerID="54f77b33f3e4c94e7917a07fb90f430bea077f4d6f12753d00817e9ed50d288d" exitCode=0 Sep 05 06:27:55.470617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.470587 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnd7c" event={"ID":"0e3bf338-be1e-4459-ac3e-a372a1696832","Type":"ContainerDied","Data":"54f77b33f3e4c94e7917a07fb90f430bea077f4d6f12753d00817e9ed50d288d"} Sep 05 06:27:55.474297 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.474269 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" event={"ID":"276312ef-5e4f-4a22-ae1c-600a2885e110","Type":"ContainerStarted","Data":"e81ccd1865ad296cb5a0accabef902a9e3b2587a793a7f96d5e8169d229010c9"} Sep 05 06:27:55.474403 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.474326 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" event={"ID":"276312ef-5e4f-4a22-ae1c-600a2885e110","Type":"ContainerStarted","Data":"2a10b866461ba2aac2095286b3f59bc50eff35fd773a524384b1b736e08b6584"} Sep 05 06:27:55.474403 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.474353 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" event={"ID":"276312ef-5e4f-4a22-ae1c-600a2885e110","Type":"ContainerStarted","Data":"8d25e663c8924330661b5cac7751eb7c7e0309b37db0c34c93c3c860646df986"} Sep 05 06:27:55.511368 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.511279 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-grpc-tls\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.513119 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.512573 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-r25d6" podStartSLOduration=1.932391393 podStartE2EDuration="3.512554079s" totalImagesPullingTime="1.580162686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:55.509093098 +0000 UTC m=+172.192598006" watchObservedRunningTime="2026-09-05 06:27:55.512554079 +0000 UTC m=+172.196058987" Sep 05 06:27:55.513259 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.513226 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.513328 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.513289 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8999492-9491-473c-a6c1-4e9ca9ee9efe-metrics-client-ca\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.518491 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.513334 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-tls\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.518491 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.513540 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.518491 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.513590 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.518491 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.513643 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.518491 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.513690 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc6lz\" (UniqueName: \"kubernetes.io/projected/b8999492-9491-473c-a6c1-4e9ca9ee9efe-kube-api-access-vc6lz\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.614983 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.614807 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.614983 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.614863 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc6lz\" (UniqueName: \"kubernetes.io/projected/b8999492-9491-473c-a6c1-4e9ca9ee9efe-kube-api-access-vc6lz\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.614983 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.614949 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-grpc-tls\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.615191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.615020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.615191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.615126 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8999492-9491-473c-a6c1-4e9ca9ee9efe-metrics-client-ca\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.615290 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.615177 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-tls\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.615290 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.615276 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.615379 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.615316 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.616129 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.616037 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8999492-9491-473c-a6c1-4e9ca9ee9efe-metrics-client-ca\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.618698 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.618228 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-grpc-tls\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.619246 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.619202 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-tls\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.620498 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.620459 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.620938 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.620917 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.621051 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.621028 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.621267 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.621249 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b8999492-9491-473c-a6c1-4e9ca9ee9efe-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.628771 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.628748 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc6lz\" (UniqueName: \"kubernetes.io/projected/b8999492-9491-473c-a6c1-4e9ca9ee9efe-kube-api-access-vc6lz\") pod \"thanos-querier-7bff8865f5-6glhx\" (UID: \"b8999492-9491-473c-a6c1-4e9ca9ee9efe\") " pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:55.734133 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:55.734096 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:27:56.226324 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:56.226300 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7bff8865f5-6glhx"] Sep 05 06:27:56.228778 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:27:56.228751 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8999492_9491_473c_a6c1_4e9ca9ee9efe.slice/crio-e87cca4c0e9557f20b2745c6e5f005319c5d61d653a9ba67bc0aeb6acba1004c WatchSource:0}: Error finding container e87cca4c0e9557f20b2745c6e5f005319c5d61d653a9ba67bc0aeb6acba1004c: Status 404 returned error can't find the container with id e87cca4c0e9557f20b2745c6e5f005319c5d61d653a9ba67bc0aeb6acba1004c Sep 05 06:27:56.479309 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:56.479220 2583 generic.go:356] "Generic (PLEG): container finished" podID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerID="c9cb5e6b723de0ee048c306f53702dd261a60419a3b624406ac544dc3228b358" exitCode=0 Sep 05 06:27:56.479452 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:56.479308 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerDied","Data":"c9cb5e6b723de0ee048c306f53702dd261a60419a3b624406ac544dc3228b358"} Sep 05 06:27:56.481861 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:56.481837 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnd7c" event={"ID":"0e3bf338-be1e-4459-ac3e-a372a1696832","Type":"ContainerStarted","Data":"0b7324e98270df7ad9c6d261940e43bf6df277f53f749526ad8a437330f06fc4"} Sep 05 06:27:56.481973 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:56.481882 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnd7c" event={"ID":"0e3bf338-be1e-4459-ac3e-a372a1696832","Type":"ContainerStarted","Data":"b4ec3b48d9ea25110612e20375e82ad0ecb720661bcac14d97144f01497b125d"} Sep 05 06:27:56.483035 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:56.483005 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" event={"ID":"b8999492-9491-473c-a6c1-4e9ca9ee9efe","Type":"ContainerStarted","Data":"e87cca4c0e9557f20b2745c6e5f005319c5d61d653a9ba67bc0aeb6acba1004c"} Sep 05 06:27:56.525133 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:56.525084 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qnd7c" podStartSLOduration=3.219037314 podStartE2EDuration="4.525069525s" totalImagesPullingTime="1.226072389s" totalInitContainerRuntime="79.959822ms" isStatefulPod=true podCreationTimestamp="2026-09-05 06:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:56.52455165 +0000 UTC m=+173.208056558" watchObservedRunningTime="2026-09-05 06:27:56.525069525 +0000 UTC m=+173.208574422" Sep 05 06:27:58.496409 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:58.496240 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerStarted","Data":"713b7c073dca36f7a97af94d1e5c10650761368c4e8b456f7eacf19b0819a6b3"} Sep 05 06:27:58.499045 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:58.498951 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" event={"ID":"b8999492-9491-473c-a6c1-4e9ca9ee9efe","Type":"ContainerStarted","Data":"e25a90764519609425eb4de6b7737804b841b18246cf4981be30658280e00bfc"} Sep 05 06:27:58.817749 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:58.817362 2583 scope.go:117] "RemoveContainer" containerID="a5c0a4ba504f91c170b75896e6f9459e888c80b89c9c019041a6c823c4f9223c" Sep 05 06:27:59.444463 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.444442 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p8fq5" Sep 05 06:27:59.512166 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.512137 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerStarted","Data":"96f7cb5ac534a01e48f7d9cfe081426d78d0c37049ae8dd83272481184ce5d0a"} Sep 05 06:27:59.512529 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.512223 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerStarted","Data":"e0a2942418285cdc6e4fec246fc79ef5f9656cd658a6ac37a0b327454e7be686"} Sep 05 06:27:59.512529 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.512265 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerStarted","Data":"1804fd019d6a0a9a7365127d6aa1bc534eed887cfdcffc9b04602ddfd766b399"} Sep 05 06:27:59.512529 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.512305 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerStarted","Data":"ecc7b770b05f1666b1372ad4698c124f79cb9e883d5905e19e3dba98a75e153e"} Sep 05 06:27:59.514732 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.514710 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:27:59.514843 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.514826 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" event={"ID":"a6b9ff6f-b0f0-43eb-8516-1939b0a95609","Type":"ContainerStarted","Data":"009ca0e73c0b79678e24da473d8c8366811ab0f1c009c7f03df05b8f872f5ac2"} Sep 05 06:27:59.515171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.515106 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:59.519400 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.519378 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" event={"ID":"b8999492-9491-473c-a6c1-4e9ca9ee9efe","Type":"ContainerStarted","Data":"75cb61b1a4146cbad0ab8a93d9fcc13db3b1881f1ffbd0f8e93f0f8940201fef"} Sep 05 06:27:59.519502 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.519443 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" event={"ID":"b8999492-9491-473c-a6c1-4e9ca9ee9efe","Type":"ContainerStarted","Data":"62d43d089fda02cf07317c4200b8694c83cebe6a079dcdc5baa41cecefc51919"} Sep 05 06:27:59.532711 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.532662 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" podStartSLOduration=42.055679167 podStartE2EDuration="44.532646402s" totalImagesPullingTime="2.476967235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:59.531714589 +0000 UTC m=+176.215219508" watchObservedRunningTime="2026-09-05 06:27:59.532646402 +0000 UTC m=+176.216151308" Sep 05 06:27:59.547218 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.547198 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-8bx7z" Sep 05 06:27:59.727012 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.726937 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-k9vlh"] Sep 05 06:27:59.730272 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.730252 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-k9vlh" Sep 05 06:27:59.732770 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.732755 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 05 06:27:59.732981 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.732958 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mlvzm\"" Sep 05 06:27:59.733090 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.733015 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 05 06:27:59.740714 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.740691 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-k9vlh"] Sep 05 06:27:59.756573 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.756549 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbplq\" (UniqueName: \"kubernetes.io/projected/367e1810-ce3c-45ab-859c-80a508dab669-kube-api-access-gbplq\") pod \"downloads-85d98c998f-k9vlh\" (UID: \"367e1810-ce3c-45ab-859c-80a508dab669\") " pod="openshift-console/downloads-85d98c998f-k9vlh" Sep 05 06:27:59.857728 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.857695 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbplq\" (UniqueName: \"kubernetes.io/projected/367e1810-ce3c-45ab-859c-80a508dab669-kube-api-access-gbplq\") pod \"downloads-85d98c998f-k9vlh\" (UID: \"367e1810-ce3c-45ab-859c-80a508dab669\") " pod="openshift-console/downloads-85d98c998f-k9vlh" Sep 05 06:27:59.868731 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:27:59.868706 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbplq\" (UniqueName: \"kubernetes.io/projected/367e1810-ce3c-45ab-859c-80a508dab669-kube-api-access-gbplq\") pod \"downloads-85d98c998f-k9vlh\" (UID: \"367e1810-ce3c-45ab-859c-80a508dab669\") " pod="openshift-console/downloads-85d98c998f-k9vlh" Sep 05 06:28:00.039872 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:00.039837 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-k9vlh" Sep 05 06:28:00.310172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:00.310144 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-k9vlh"] Sep 05 06:28:00.312688 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:28:00.312664 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod367e1810_ce3c_45ab_859c_80a508dab669.slice/crio-1db4025435e1c90d2b7d3d65f42027ffa5494993246576fb790b93432acd8408 WatchSource:0}: Error finding container 1db4025435e1c90d2b7d3d65f42027ffa5494993246576fb790b93432acd8408: Status 404 returned error can't find the container with id 1db4025435e1c90d2b7d3d65f42027ffa5494993246576fb790b93432acd8408 Sep 05 06:28:00.526541 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:00.526507 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" event={"ID":"b8999492-9491-473c-a6c1-4e9ca9ee9efe","Type":"ContainerStarted","Data":"3af22a9f6ebd06264159111745ba7e899fcdb227bf089bbeda4f649753a7a8f9"} Sep 05 06:28:00.526951 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:00.526582 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" event={"ID":"b8999492-9491-473c-a6c1-4e9ca9ee9efe","Type":"ContainerStarted","Data":"51cfa9d6575fe06c838ed367a1841ace2078725741aa50bd7294f0c543df08a5"} Sep 05 06:28:00.526951 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:00.526612 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" event={"ID":"b8999492-9491-473c-a6c1-4e9ca9ee9efe","Type":"ContainerStarted","Data":"fe856817430391027e6421a7316222c2ce202efdfeed908f607820fb10d9a152"} Sep 05 06:28:00.526951 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:00.526744 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:28:00.527802 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:00.527777 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-k9vlh" event={"ID":"367e1810-ce3c-45ab-859c-80a508dab669","Type":"ContainerStarted","Data":"1db4025435e1c90d2b7d3d65f42027ffa5494993246576fb790b93432acd8408"} Sep 05 06:28:00.531110 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:00.531089 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerStarted","Data":"a659a486221ac245af29ff7e4a846ae462634628c21a2caaa4d4577ae5634586"} Sep 05 06:28:00.552854 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:00.552812 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" podStartSLOduration=2.569003219 podStartE2EDuration="5.552799384s" totalImagesPullingTime="2.983796165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:00.550550232 +0000 UTC m=+177.234055139" watchObservedRunningTime="2026-09-05 06:28:00.552799384 +0000 UTC m=+177.236304290" Sep 05 06:28:00.581998 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:00.581911 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.988623296 podStartE2EDuration="7.581894399s" totalImagesPullingTime="3.578320302s" totalInitContainerRuntime="14.950801ms" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:00.580717001 +0000 UTC m=+177.264221934" watchObservedRunningTime="2026-09-05 06:28:00.581894399 +0000 UTC m=+177.265399305" Sep 05 06:28:06.543281 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:06.543244 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7bff8865f5-6glhx" Sep 05 06:28:09.158745 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.158709 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c6868fd5-s7xsq"] Sep 05 06:28:09.164947 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.164922 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.168252 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.168229 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 05 06:28:09.169547 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.169376 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 05 06:28:09.169547 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.169412 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 05 06:28:09.169817 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.169793 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 05 06:28:09.170200 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.170173 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 05 06:28:09.170200 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.170173 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-n9z8k\"" Sep 05 06:28:09.180742 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.180720 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6868fd5-s7xsq"] Sep 05 06:28:09.248347 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.248318 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-serving-cert\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.248508 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.248387 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-oauth-serving-cert\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.248508 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.248494 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-service-ca\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.248604 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.248528 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjhxc\" (UniqueName: \"kubernetes.io/projected/36a2e258-d23d-4199-be97-a49361fbe5f4-kube-api-access-xjhxc\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.248645 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.248595 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-console-config\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.248699 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.248670 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-oauth-config\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.349633 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.349599 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-serving-cert\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.349790 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.349663 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-oauth-serving-cert\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.349790 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.349704 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-service-ca\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.349790 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.349726 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhxc\" (UniqueName: \"kubernetes.io/projected/36a2e258-d23d-4199-be97-a49361fbe5f4-kube-api-access-xjhxc\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.349790 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.349757 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-console-config\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.350008 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.349809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-oauth-config\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.350580 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.350551 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-oauth-serving-cert\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.350704 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.350551 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-service-ca\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.350764 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.350744 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-console-config\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.352396 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.352372 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-oauth-config\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.352573 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.352554 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-serving-cert\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.359382 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.359363 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjhxc\" (UniqueName: \"kubernetes.io/projected/36a2e258-d23d-4199-be97-a49361fbe5f4-kube-api-access-xjhxc\") pod \"console-c6868fd5-s7xsq\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:09.476671 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:09.476579 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:15.832467 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.832408 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7554759b8-cp67b"] Sep 05 06:28:15.837610 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.837586 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:15.853018 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.848833 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7554759b8-cp67b"] Sep 05 06:28:15.853018 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.849226 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 05 06:28:15.907534 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.907504 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-trusted-ca-bundle\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:15.907685 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.907543 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k6kv\" (UniqueName: \"kubernetes.io/projected/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-kube-api-access-7k6kv\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:15.907685 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.907567 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-serving-cert\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:15.907685 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.907650 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-config\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:15.907833 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.907686 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-service-ca\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:15.907833 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.907716 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-oauth-serving-cert\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:15.907918 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:15.907850 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-oauth-config\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.009117 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.009087 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-serving-cert\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.009252 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.009141 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-config\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.009252 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.009166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-service-ca\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.009252 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.009190 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-oauth-serving-cert\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.009252 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.009249 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-oauth-config\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.009444 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.009287 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-trusted-ca-bundle\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.009520 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.009454 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k6kv\" (UniqueName: \"kubernetes.io/projected/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-kube-api-access-7k6kv\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.010312 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.010282 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-service-ca\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.010416 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.010310 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-oauth-serving-cert\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.010515 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.010472 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-config\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.010627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.010607 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-trusted-ca-bundle\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.011888 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.011862 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-oauth-config\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.012071 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.012050 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-serving-cert\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.018326 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.018305 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k6kv\" (UniqueName: \"kubernetes.io/projected/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-kube-api-access-7k6kv\") pod \"console-7554759b8-cp67b\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.151444 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.151362 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:16.560022 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.559827 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7554759b8-cp67b"] Sep 05 06:28:16.562329 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.562303 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6868fd5-s7xsq"] Sep 05 06:28:16.562896 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:28:16.562861 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a4d02a8_cd62_47c4_99bc_78dfbce85a56.slice/crio-23db67ee644231b9d05f07530dbacf953c014377d36324a050dded8d1fa1093e WatchSource:0}: Error finding container 23db67ee644231b9d05f07530dbacf953c014377d36324a050dded8d1fa1093e: Status 404 returned error can't find the container with id 23db67ee644231b9d05f07530dbacf953c014377d36324a050dded8d1fa1093e Sep 05 06:28:16.566595 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:28:16.566550 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a2e258_d23d_4199_be97_a49361fbe5f4.slice/crio-4428492a47de28f536d807588dd8d2dc1526317333955b420e8d1f86dd839c0b WatchSource:0}: Error finding container 4428492a47de28f536d807588dd8d2dc1526317333955b420e8d1f86dd839c0b: Status 404 returned error can't find the container with id 4428492a47de28f536d807588dd8d2dc1526317333955b420e8d1f86dd839c0b Sep 05 06:28:16.607074 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.607037 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-k9vlh" event={"ID":"367e1810-ce3c-45ab-859c-80a508dab669","Type":"ContainerStarted","Data":"a10b83d3991bbfd6228e0d7fc66a5650913f080dc5eb57c90cc7cbf1ca49f552"} Sep 05 06:28:16.607286 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.607260 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-k9vlh" Sep 05 06:28:16.608729 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.608705 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6868fd5-s7xsq" event={"ID":"36a2e258-d23d-4199-be97-a49361fbe5f4","Type":"ContainerStarted","Data":"4428492a47de28f536d807588dd8d2dc1526317333955b420e8d1f86dd839c0b"} Sep 05 06:28:16.610112 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.610076 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7554759b8-cp67b" event={"ID":"9a4d02a8-cd62-47c4-99bc-78dfbce85a56","Type":"ContainerStarted","Data":"23db67ee644231b9d05f07530dbacf953c014377d36324a050dded8d1fa1093e"} Sep 05 06:28:16.616876 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.616848 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-k9vlh" Sep 05 06:28:16.625351 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:16.625307 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-k9vlh" podStartSLOduration=1.5547594409999999 podStartE2EDuration="17.625292775s" totalImagesPullingTime="16.070533334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:16.624637135 +0000 UTC m=+193.308142043" watchObservedRunningTime="2026-09-05 06:28:16.625292775 +0000 UTC m=+193.308797682" Sep 05 06:28:21.647869 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:21.647825 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7554759b8-cp67b" event={"ID":"9a4d02a8-cd62-47c4-99bc-78dfbce85a56","Type":"ContainerStarted","Data":"1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6"} Sep 05 06:28:21.649656 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:21.649627 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6868fd5-s7xsq" event={"ID":"36a2e258-d23d-4199-be97-a49361fbe5f4","Type":"ContainerStarted","Data":"f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e"} Sep 05 06:28:21.669298 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:21.669239 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7554759b8-cp67b" podStartSLOduration=2.392487704 podStartE2EDuration="6.669218815s" totalImagesPullingTime="4.276731111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:21.66769527 +0000 UTC m=+198.351200178" watchObservedRunningTime="2026-09-05 06:28:21.669218815 +0000 UTC m=+198.352723723" Sep 05 06:28:21.691196 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:21.691147 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c6868fd5-s7xsq" podStartSLOduration=8.410228061 podStartE2EDuration="12.691128291s" totalImagesPullingTime="4.28090023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:21.688613012 +0000 UTC m=+198.372117919" watchObservedRunningTime="2026-09-05 06:28:21.691128291 +0000 UTC m=+198.374633198" Sep 05 06:28:26.152060 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:26.152026 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:26.152538 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:26.152082 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:26.156726 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:26.156707 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:26.675793 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:26.675763 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:28:26.743248 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:26.743209 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6868fd5-s7xsq"] Sep 05 06:28:27.366401 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:27.366370 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_689e8836-47ec-4bb4-9d1a-90c0c43c48bc/init-config-reloader/0.log" Sep 05 06:28:27.564446 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:27.564419 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_689e8836-47ec-4bb4-9d1a-90c0c43c48bc/alertmanager/0.log" Sep 05 06:28:27.770202 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:27.770177 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_689e8836-47ec-4bb4-9d1a-90c0c43c48bc/config-reloader/0.log" Sep 05 06:28:27.963641 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:27.963613 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_689e8836-47ec-4bb4-9d1a-90c0c43c48bc/kube-rbac-proxy-web/0.log" Sep 05 06:28:28.168449 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:28.168383 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_689e8836-47ec-4bb4-9d1a-90c0c43c48bc/kube-rbac-proxy/0.log" Sep 05 06:28:28.371264 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:28.371235 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_689e8836-47ec-4bb4-9d1a-90c0c43c48bc/kube-rbac-proxy-metric/0.log" Sep 05 06:28:28.563150 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:28.563123 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_689e8836-47ec-4bb4-9d1a-90c0c43c48bc/prom-label-proxy/0.log" Sep 05 06:28:28.765136 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:28.765101 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-6r7ml_2c15fd53-f580-45ab-aacf-5a21e7500817/cluster-monitoring-operator/0.log" Sep 05 06:28:28.969132 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:28.969031 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-r25d6_276312ef-5e4f-4a22-ae1c-600a2885e110/kube-state-metrics/0.log" Sep 05 06:28:29.174927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:29.174889 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-r25d6_276312ef-5e4f-4a22-ae1c-600a2885e110/kube-rbac-proxy-main/0.log" Sep 05 06:28:29.364626 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:29.364546 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-r25d6_276312ef-5e4f-4a22-ae1c-600a2885e110/kube-rbac-proxy-self/0.log" Sep 05 06:28:29.477579 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:29.477532 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:30.563429 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:30.563398 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnd7c_0e3bf338-be1e-4459-ac3e-a372a1696832/init-textfile/0.log" Sep 05 06:28:30.765744 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:30.765716 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnd7c_0e3bf338-be1e-4459-ac3e-a372a1696832/node-exporter/0.log" Sep 05 06:28:30.964434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:30.964335 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnd7c_0e3bf338-be1e-4459-ac3e-a372a1696832/kube-rbac-proxy/0.log" Sep 05 06:28:33.764278 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:33.764249 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-fwzxk_a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451/prometheus-operator/0.log" Sep 05 06:28:33.963806 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:33.963775 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-fwzxk_a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451/kube-rbac-proxy/0.log" Sep 05 06:28:34.165240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:34.165159 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-5cq9h_32e578ed-c724-41ac-9734-d82e91cfefb2/prometheus-operator-admission-webhook/0.log" Sep 05 06:28:34.363668 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:34.363615 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/thanos-query/0.log" Sep 05 06:28:34.564778 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:34.564748 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/kube-rbac-proxy-web/0.log" Sep 05 06:28:34.764605 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:34.764573 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/kube-rbac-proxy/0.log" Sep 05 06:28:34.965154 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:34.965084 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/prom-label-proxy/0.log" Sep 05 06:28:35.165050 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:35.165024 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/kube-rbac-proxy-rules/0.log" Sep 05 06:28:35.366204 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:35.366174 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/kube-rbac-proxy-metrics/0.log" Sep 05 06:28:35.564197 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:35.564142 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-fwvvg_d115bb76-6396-40b9-b87a-1e1ab8d31282/networking-console-plugin/0.log" Sep 05 06:28:35.763881 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:35.763856 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:28:35.965102 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:35.965076 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/3.log" Sep 05 06:28:36.163672 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:36.163605 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7554759b8-cp67b_9a4d02a8-cd62-47c4-99bc-78dfbce85a56/console/0.log" Sep 05 06:28:36.365176 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:36.365149 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-c6868fd5-s7xsq_36a2e258-d23d-4199-be97-a49361fbe5f4/console/0.log" Sep 05 06:28:36.565906 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:36.565872 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-k9vlh_367e1810-ce3c-45ab-859c-80a508dab669/download-server/0.log" Sep 05 06:28:37.164665 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:37.164636 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hwz4f_2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4/serve-healthcheck-canary/0.log" Sep 05 06:28:39.725344 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:39.725241 2583 generic.go:356] "Generic (PLEG): container finished" podID="ad94c7a8-4641-4f4d-b7fa-56e32c098ee8" containerID="cc4270c3c3fad2dac401f5b76110e786e6b8c7f58a7f838e2af60f8be31bcd8e" exitCode=0 Sep 05 06:28:39.725344 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:39.725316 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" event={"ID":"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8","Type":"ContainerDied","Data":"cc4270c3c3fad2dac401f5b76110e786e6b8c7f58a7f838e2af60f8be31bcd8e"} Sep 05 06:28:39.725918 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:39.725704 2583 scope.go:117] "RemoveContainer" containerID="cc4270c3c3fad2dac401f5b76110e786e6b8c7f58a7f838e2af60f8be31bcd8e" Sep 05 06:28:40.735357 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:40.735313 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-nj2m5" event={"ID":"ad94c7a8-4641-4f4d-b7fa-56e32c098ee8","Type":"ContainerStarted","Data":"7932f705c1a3ada0a46b7a30ef57ca40312bdfb99848de604366676c5e617546"} Sep 05 06:28:51.764363 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:51.764310 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c6868fd5-s7xsq" podUID="36a2e258-d23d-4199-be97-a49361fbe5f4" containerName="console" containerID="cri-o://f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e" gracePeriod=15 Sep 05 06:28:52.036432 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.036366 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:52.133025 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.132992 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-console-config\") pod \"36a2e258-d23d-4199-be97-a49361fbe5f4\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " Sep 05 06:28:52.133170 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.133071 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-serving-cert\") pod \"36a2e258-d23d-4199-be97-a49361fbe5f4\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " Sep 05 06:28:52.133170 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.133122 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-service-ca\") pod \"36a2e258-d23d-4199-be97-a49361fbe5f4\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " Sep 05 06:28:52.133170 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.133145 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-oauth-serving-cert\") pod \"36a2e258-d23d-4199-be97-a49361fbe5f4\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " Sep 05 06:28:52.133170 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.133162 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-oauth-config\") pod \"36a2e258-d23d-4199-be97-a49361fbe5f4\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " Sep 05 06:28:52.133366 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.133188 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjhxc\" (UniqueName: \"kubernetes.io/projected/36a2e258-d23d-4199-be97-a49361fbe5f4-kube-api-access-xjhxc\") pod \"36a2e258-d23d-4199-be97-a49361fbe5f4\" (UID: \"36a2e258-d23d-4199-be97-a49361fbe5f4\") " Sep 05 06:28:52.133520 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.133492 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-console-config" (OuterVolumeSpecName: "console-config") pod "36a2e258-d23d-4199-be97-a49361fbe5f4" (UID: "36a2e258-d23d-4199-be97-a49361fbe5f4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:28:52.133594 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.133516 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "36a2e258-d23d-4199-be97-a49361fbe5f4" (UID: "36a2e258-d23d-4199-be97-a49361fbe5f4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:28:52.133648 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.133590 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-service-ca" (OuterVolumeSpecName: "service-ca") pod "36a2e258-d23d-4199-be97-a49361fbe5f4" (UID: "36a2e258-d23d-4199-be97-a49361fbe5f4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:28:52.135300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.135278 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "36a2e258-d23d-4199-be97-a49361fbe5f4" (UID: "36a2e258-d23d-4199-be97-a49361fbe5f4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:28:52.135381 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.135330 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36a2e258-d23d-4199-be97-a49361fbe5f4-kube-api-access-xjhxc" (OuterVolumeSpecName: "kube-api-access-xjhxc") pod "36a2e258-d23d-4199-be97-a49361fbe5f4" (UID: "36a2e258-d23d-4199-be97-a49361fbe5f4"). InnerVolumeSpecName "kube-api-access-xjhxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:28:52.135381 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.135363 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "36a2e258-d23d-4199-be97-a49361fbe5f4" (UID: "36a2e258-d23d-4199-be97-a49361fbe5f4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:28:52.234450 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.234423 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-console-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:28:52.234450 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.234444 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-serving-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:28:52.234450 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.234456 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:28:52.234651 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.234465 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36a2e258-d23d-4199-be97-a49361fbe5f4-oauth-serving-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:28:52.234651 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.234488 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36a2e258-d23d-4199-be97-a49361fbe5f4-console-oauth-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:28:52.234651 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.234497 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjhxc\" (UniqueName: \"kubernetes.io/projected/36a2e258-d23d-4199-be97-a49361fbe5f4-kube-api-access-xjhxc\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:28:52.786076 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.786044 2583 generic.go:356] "Generic (PLEG): container finished" podID="36a2e258-d23d-4199-be97-a49361fbe5f4" containerID="f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e" exitCode=0 Sep 05 06:28:52.786501 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.786107 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6868fd5-s7xsq" Sep 05 06:28:52.786501 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.786134 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6868fd5-s7xsq" event={"ID":"36a2e258-d23d-4199-be97-a49361fbe5f4","Type":"ContainerDied","Data":"f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e"} Sep 05 06:28:52.786501 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.786186 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6868fd5-s7xsq" event={"ID":"36a2e258-d23d-4199-be97-a49361fbe5f4","Type":"ContainerDied","Data":"4428492a47de28f536d807588dd8d2dc1526317333955b420e8d1f86dd839c0b"} Sep 05 06:28:52.786501 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.786207 2583 scope.go:117] "RemoveContainer" containerID="f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e" Sep 05 06:28:52.794910 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.794870 2583 scope.go:117] "RemoveContainer" containerID="f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e" Sep 05 06:28:52.795177 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:28:52.795159 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e\": container with ID starting with f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e not found: ID does not exist" containerID="f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e" Sep 05 06:28:52.795738 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.795185 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e"} err="failed to get container status \"f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e\": rpc error: code = NotFound desc = could not find container \"f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e\": container with ID starting with f9a4d1fc915629ee3a7412117dc7c51362230aa07257c2187d2f09e76210b76e not found: ID does not exist" Sep 05 06:28:52.809036 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.809012 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c6868fd5-s7xsq"] Sep 05 06:28:52.812836 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:52.812818 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c6868fd5-s7xsq"] Sep 05 06:28:53.824029 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:28:53.823993 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36a2e258-d23d-4199-be97-a49361fbe5f4" path="/var/lib/kubelet/pods/36a2e258-d23d-4199-be97-a49361fbe5f4/volumes" Sep 05 06:29:12.703206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.703171 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 05 06:29:12.703711 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.703465 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="alertmanager" containerID="cri-o://713b7c073dca36f7a97af94d1e5c10650761368c4e8b456f7eacf19b0819a6b3" gracePeriod=120 Sep 05 06:29:12.703711 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.703552 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy-web" containerID="cri-o://1804fd019d6a0a9a7365127d6aa1bc534eed887cfdcffc9b04602ddfd766b399" gracePeriod=120 Sep 05 06:29:12.703711 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.703602 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="prom-label-proxy" containerID="cri-o://a659a486221ac245af29ff7e4a846ae462634628c21a2caaa4d4577ae5634586" gracePeriod=120 Sep 05 06:29:12.703711 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.703601 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy" containerID="cri-o://e0a2942418285cdc6e4fec246fc79ef5f9656cd658a6ac37a0b327454e7be686" gracePeriod=120 Sep 05 06:29:12.703711 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.703552 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy-metric" containerID="cri-o://96f7cb5ac534a01e48f7d9cfe081426d78d0c37049ae8dd83272481184ce5d0a" gracePeriod=120 Sep 05 06:29:12.703986 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.703601 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="config-reloader" containerID="cri-o://ecc7b770b05f1666b1372ad4698c124f79cb9e883d5905e19e3dba98a75e153e" gracePeriod=120 Sep 05 06:29:12.874564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.874531 2583 generic.go:356] "Generic (PLEG): container finished" podID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerID="a659a486221ac245af29ff7e4a846ae462634628c21a2caaa4d4577ae5634586" exitCode=0 Sep 05 06:29:12.874564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.874554 2583 generic.go:356] "Generic (PLEG): container finished" podID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerID="e0a2942418285cdc6e4fec246fc79ef5f9656cd658a6ac37a0b327454e7be686" exitCode=0 Sep 05 06:29:12.874564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.874562 2583 generic.go:356] "Generic (PLEG): container finished" podID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerID="ecc7b770b05f1666b1372ad4698c124f79cb9e883d5905e19e3dba98a75e153e" exitCode=0 Sep 05 06:29:12.874564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.874568 2583 generic.go:356] "Generic (PLEG): container finished" podID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerID="713b7c073dca36f7a97af94d1e5c10650761368c4e8b456f7eacf19b0819a6b3" exitCode=0 Sep 05 06:29:12.874761 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.874600 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerDied","Data":"a659a486221ac245af29ff7e4a846ae462634628c21a2caaa4d4577ae5634586"} Sep 05 06:29:12.874761 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.874668 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerDied","Data":"e0a2942418285cdc6e4fec246fc79ef5f9656cd658a6ac37a0b327454e7be686"} Sep 05 06:29:12.874761 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.874696 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerDied","Data":"ecc7b770b05f1666b1372ad4698c124f79cb9e883d5905e19e3dba98a75e153e"} Sep 05 06:29:12.874761 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:12.874720 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerDied","Data":"713b7c073dca36f7a97af94d1e5c10650761368c4e8b456f7eacf19b0819a6b3"} Sep 05 06:29:13.883572 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:13.883541 2583 generic.go:356] "Generic (PLEG): container finished" podID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerID="96f7cb5ac534a01e48f7d9cfe081426d78d0c37049ae8dd83272481184ce5d0a" exitCode=0 Sep 05 06:29:13.883572 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:13.883567 2583 generic.go:356] "Generic (PLEG): container finished" podID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerID="1804fd019d6a0a9a7365127d6aa1bc534eed887cfdcffc9b04602ddfd766b399" exitCode=0 Sep 05 06:29:13.884025 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:13.883614 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerDied","Data":"96f7cb5ac534a01e48f7d9cfe081426d78d0c37049ae8dd83272481184ce5d0a"} Sep 05 06:29:13.884025 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:13.883688 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerDied","Data":"1804fd019d6a0a9a7365127d6aa1bc534eed887cfdcffc9b04602ddfd766b399"} Sep 05 06:29:13.947649 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:13.947626 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:14.118722 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.118642 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62988\" (UniqueName: \"kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-kube-api-access-62988\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.118722 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.118702 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-volume\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.118889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.118731 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-tls-assets\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.118889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.118755 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.118889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.118784 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-cluster-tls-config\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.118889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.118831 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-trusted-ca-bundle\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.118889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.118868 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-web\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.119149 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.118938 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.119149 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.118976 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-web-config\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.119149 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.118999 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-main-tls\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.119149 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.119022 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-metrics-client-ca\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.119149 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.119048 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-out\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.119149 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.119077 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-main-db\") pod \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\" (UID: \"689e8836-47ec-4bb4-9d1a-90c0c43c48bc\") " Sep 05 06:29:14.119448 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.119246 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:29:14.119448 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.119356 2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.120690 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.120617 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:29:14.120952 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.120924 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:29:14.121559 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.121526 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-kube-api-access-62988" (OuterVolumeSpecName: "kube-api-access-62988") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "kube-api-access-62988". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:29:14.121686 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.121566 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:29:14.121766 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.121697 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:29:14.121766 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.121735 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-volume" (OuterVolumeSpecName: "config-volume") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:29:14.122286 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.122253 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:29:14.122610 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.122585 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:29:14.123030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.123007 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:29:14.123472 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.123451 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-out" (OuterVolumeSpecName: "config-out") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:29:14.126216 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.126196 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:29:14.133196 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.133176 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-web-config" (OuterVolumeSpecName: "web-config") pod "689e8836-47ec-4bb4-9d1a-90c0c43c48bc" (UID: "689e8836-47ec-4bb4-9d1a-90c0c43c48bc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:29:14.219914 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219881 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62988\" (UniqueName: \"kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-kube-api-access-62988\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.219914 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219911 2583 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-volume\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.219914 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219921 2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-tls-assets\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.220062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219930 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.220062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219939 2583 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-cluster-tls-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.220062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219949 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.220062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219958 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.220062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219967 2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-web-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.220062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219976 2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-secret-alertmanager-main-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.220062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219984 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-metrics-client-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.220062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.219994 2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-config-out\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.220062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.220001 2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/689e8836-47ec-4bb4-9d1a-90c0c43c48bc-alertmanager-main-db\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:14.890667 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.890639 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:14.891032 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.890641 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"689e8836-47ec-4bb4-9d1a-90c0c43c48bc","Type":"ContainerDied","Data":"5054cd8b70d4f474a20f92b7d7022dee1ba2ba6c2e8f564bb943fdc82df8bdf0"} Sep 05 06:29:14.891032 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.890781 2583 scope.go:117] "RemoveContainer" containerID="a659a486221ac245af29ff7e4a846ae462634628c21a2caaa4d4577ae5634586" Sep 05 06:29:14.898512 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.898492 2583 scope.go:117] "RemoveContainer" containerID="96f7cb5ac534a01e48f7d9cfe081426d78d0c37049ae8dd83272481184ce5d0a" Sep 05 06:29:14.905918 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.905900 2583 scope.go:117] "RemoveContainer" containerID="e0a2942418285cdc6e4fec246fc79ef5f9656cd658a6ac37a0b327454e7be686" Sep 05 06:29:14.913757 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.913697 2583 scope.go:117] "RemoveContainer" containerID="1804fd019d6a0a9a7365127d6aa1bc534eed887cfdcffc9b04602ddfd766b399" Sep 05 06:29:14.915799 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.915777 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 05 06:29:14.920992 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.920963 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 05 06:29:14.921430 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.921402 2583 scope.go:117] "RemoveContainer" containerID="ecc7b770b05f1666b1372ad4698c124f79cb9e883d5905e19e3dba98a75e153e" Sep 05 06:29:14.928639 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.928620 2583 scope.go:117] "RemoveContainer" containerID="713b7c073dca36f7a97af94d1e5c10650761368c4e8b456f7eacf19b0819a6b3" Sep 05 06:29:14.935400 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.935386 2583 scope.go:117] "RemoveContainer" containerID="c9cb5e6b723de0ee048c306f53702dd261a60419a3b624406ac544dc3228b358" Sep 05 06:29:14.948726 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.948708 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 05 06:29:14.948993 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.948982 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="config-reloader" Sep 05 06:29:14.949036 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.948995 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="config-reloader" Sep 05 06:29:14.949036 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949003 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a2e258-d23d-4199-be97-a49361fbe5f4" containerName="console" Sep 05 06:29:14.949036 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949008 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a2e258-d23d-4199-be97-a49361fbe5f4" containerName="console" Sep 05 06:29:14.949036 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949019 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="init-config-reloader" Sep 05 06:29:14.949036 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949025 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="init-config-reloader" Sep 05 06:29:14.949036 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949033 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy-web" Sep 05 06:29:14.949036 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949038 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy-web" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949045 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="alertmanager" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949050 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="alertmanager" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949057 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="prom-label-proxy" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949062 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="prom-label-proxy" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949069 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949074 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949079 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy-metric" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949084 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy-metric" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949135 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy-metric" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949143 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a2e258-d23d-4199-be97-a49361fbe5f4" containerName="console" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949150 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy-web" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949156 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="prom-label-proxy" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949163 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="kube-rbac-proxy" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949169 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="alertmanager" Sep 05 06:29:14.949235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.949174 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" containerName="config-reloader" Sep 05 06:29:14.952499 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.952463 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:14.955008 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.954985 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 05 06:29:14.955105 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.955043 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 05 06:29:14.955105 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.955065 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 05 06:29:14.955105 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.955065 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 05 06:29:14.955260 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.955103 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 05 06:29:14.955674 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.955651 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 05 06:29:14.955762 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.955710 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qbfr2\"" Sep 05 06:29:14.956399 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.956210 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 05 06:29:14.956399 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.956325 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 05 06:29:14.961391 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.961367 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 05 06:29:14.968438 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:14.968416 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 05 06:29:15.026044 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c71c9f8-6b23-4ac4-9d26-4195100e246d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026044 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026044 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c71c9f8-6b23-4ac4-9d26-4195100e246d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026071 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026120 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-web-config\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026158 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d69m\" (UniqueName: \"kubernetes.io/projected/6c71c9f8-6b23-4ac4-9d26-4195100e246d-kube-api-access-9d69m\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026192 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026217 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c71c9f8-6b23-4ac4-9d26-4195100e246d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026410 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026262 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c71c9f8-6b23-4ac4-9d26-4195100e246d-config-out\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026410 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026278 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026410 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026332 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026410 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026351 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c71c9f8-6b23-4ac4-9d26-4195100e246d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026410 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026375 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.026410 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.026401 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127098 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127024 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127098 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127054 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c71c9f8-6b23-4ac4-9d26-4195100e246d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127098 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127073 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127103 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127136 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c71c9f8-6b23-4ac4-9d26-4195100e246d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127158 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c71c9f8-6b23-4ac4-9d26-4195100e246d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127193 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127218 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-web-config\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127242 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d69m\" (UniqueName: \"kubernetes.io/projected/6c71c9f8-6b23-4ac4-9d26-4195100e246d-kube-api-access-9d69m\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127284 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c71c9f8-6b23-4ac4-9d26-4195100e246d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127326 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c71c9f8-6b23-4ac4-9d26-4195100e246d-config-out\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.127876 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.127350 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.128467 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.128174 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c71c9f8-6b23-4ac4-9d26-4195100e246d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.128467 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.128319 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6c71c9f8-6b23-4ac4-9d26-4195100e246d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.129854 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.129831 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c71c9f8-6b23-4ac4-9d26-4195100e246d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.130167 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.130143 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6c71c9f8-6b23-4ac4-9d26-4195100e246d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.130651 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.130390 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.130651 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.130434 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.130651 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.130612 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-web-config\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.130854 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.130836 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6c71c9f8-6b23-4ac4-9d26-4195100e246d-config-out\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.130914 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.130885 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.131074 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.131042 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.131841 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.131822 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-config-volume\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.131966 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.131948 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6c71c9f8-6b23-4ac4-9d26-4195100e246d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.137440 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.137415 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d69m\" (UniqueName: \"kubernetes.io/projected/6c71c9f8-6b23-4ac4-9d26-4195100e246d-kube-api-access-9d69m\") pod \"alertmanager-main-0\" (UID: \"6c71c9f8-6b23-4ac4-9d26-4195100e246d\") " pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.268348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.268325 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 05 06:29:15.545257 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.545225 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 05 06:29:15.548432 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:29:15.548401 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c71c9f8_6b23_4ac4_9d26_4195100e246d.slice/crio-3371795d6fb84fcc6d8b521ba69cb05c4cbccb0bcf62dc2b6460a24fe1b5f4a9 WatchSource:0}: Error finding container 3371795d6fb84fcc6d8b521ba69cb05c4cbccb0bcf62dc2b6460a24fe1b5f4a9: Status 404 returned error can't find the container with id 3371795d6fb84fcc6d8b521ba69cb05c4cbccb0bcf62dc2b6460a24fe1b5f4a9 Sep 05 06:29:15.631848 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.631826 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:29:15.633800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.633779 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68080680-0551-487c-80ed-b93acb2f68f8-metrics-certs\") pod \"network-metrics-daemon-m8fn4\" (UID: \"68080680-0551-487c-80ed-b93acb2f68f8\") " pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:29:15.721646 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.721624 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6v7qh\"" Sep 05 06:29:15.729823 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.729803 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m8fn4" Sep 05 06:29:15.821365 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.821297 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="689e8836-47ec-4bb4-9d1a-90c0c43c48bc" path="/var/lib/kubelet/pods/689e8836-47ec-4bb4-9d1a-90c0c43c48bc/volumes" Sep 05 06:29:15.897271 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.897238 2583 generic.go:356] "Generic (PLEG): container finished" podID="6c71c9f8-6b23-4ac4-9d26-4195100e246d" containerID="9d47190c43fbaebec3cc400e6dd345a0728a3eae76c2a39c77292334c298f420" exitCode=0 Sep 05 06:29:15.897624 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.897285 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c71c9f8-6b23-4ac4-9d26-4195100e246d","Type":"ContainerDied","Data":"9d47190c43fbaebec3cc400e6dd345a0728a3eae76c2a39c77292334c298f420"} Sep 05 06:29:15.897624 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.897344 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c71c9f8-6b23-4ac4-9d26-4195100e246d","Type":"ContainerStarted","Data":"3371795d6fb84fcc6d8b521ba69cb05c4cbccb0bcf62dc2b6460a24fe1b5f4a9"} Sep 05 06:29:15.996372 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:15.996344 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m8fn4"] Sep 05 06:29:16.001533 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:29:16.001304 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68080680_0551_487c_80ed_b93acb2f68f8.slice/crio-0d2aa2ecd57320f605ad1096cbcf8d4f71acd29ec6d31a76b09702c3f969ad7a WatchSource:0}: Error finding container 0d2aa2ecd57320f605ad1096cbcf8d4f71acd29ec6d31a76b09702c3f969ad7a: Status 404 returned error can't find the container with id 0d2aa2ecd57320f605ad1096cbcf8d4f71acd29ec6d31a76b09702c3f969ad7a Sep 05 06:29:16.757015 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.756976 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-b47c7d785-t5d9g"] Sep 05 06:29:16.759870 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.759841 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.762681 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.762651 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-j6npf\"" Sep 05 06:29:16.762814 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.762721 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 05 06:29:16.762814 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.762652 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 05 06:29:16.762814 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.762769 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 05 06:29:16.766011 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.765144 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 05 06:29:16.766203 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.766182 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 05 06:29:16.769868 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.769840 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 05 06:29:16.775787 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.775763 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b47c7d785-t5d9g"] Sep 05 06:29:16.842245 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.842199 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-telemeter-client-tls\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.842245 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.842244 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcwbr\" (UniqueName: \"kubernetes.io/projected/47e4b646-b3ad-448c-b934-278d1b7b2d59-kube-api-access-rcwbr\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.842466 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.842287 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-federate-client-tls\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.842466 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.842371 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.842660 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.842458 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-secret-telemeter-client\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.842660 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.842530 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e4b646-b3ad-448c-b934-278d1b7b2d59-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.842660 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.842566 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e4b646-b3ad-448c-b934-278d1b7b2d59-metrics-client-ca\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.842660 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.842607 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e4b646-b3ad-448c-b934-278d1b7b2d59-serving-certs-ca-bundle\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.908811 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.908645 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c71c9f8-6b23-4ac4-9d26-4195100e246d","Type":"ContainerStarted","Data":"7dbc323db9ad87e0cedbfafbe1b9ba9b5757a0a9e7e1e2bfd4fa893f626c5b20"} Sep 05 06:29:16.908811 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.908729 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c71c9f8-6b23-4ac4-9d26-4195100e246d","Type":"ContainerStarted","Data":"6a86f0b66205a7ed24d94df28d207653f694f232530d679657acdb4a1740d117"} Sep 05 06:29:16.908811 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.908772 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c71c9f8-6b23-4ac4-9d26-4195100e246d","Type":"ContainerStarted","Data":"5e9dd6275a47d8e9261ac0ceacfa79a1a361e7bcd795be97d48cf6d1eb876b2a"} Sep 05 06:29:16.909371 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.908810 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c71c9f8-6b23-4ac4-9d26-4195100e246d","Type":"ContainerStarted","Data":"8964610e9e095057a161e9748ded5b069dad8fa5131fb237d17cd2e59f0bb86e"} Sep 05 06:29:16.909371 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.908860 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c71c9f8-6b23-4ac4-9d26-4195100e246d","Type":"ContainerStarted","Data":"a7a1309ca0a84f025d293bf1dc0df644889336b3d3a984f03975c36002e54a2f"} Sep 05 06:29:16.909371 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.908897 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6c71c9f8-6b23-4ac4-9d26-4195100e246d","Type":"ContainerStarted","Data":"d9f77ac69d3ce51527ceaa3a1f93d8dd2105feabf854d14824245ce9325f9890"} Sep 05 06:29:16.910128 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.910099 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m8fn4" event={"ID":"68080680-0551-487c-80ed-b93acb2f68f8","Type":"ContainerStarted","Data":"0d2aa2ecd57320f605ad1096cbcf8d4f71acd29ec6d31a76b09702c3f969ad7a"} Sep 05 06:29:16.939216 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.939166 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.924975979 podStartE2EDuration="2.939148108s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.172129ms" isStatefulPod=false podCreationTimestamp="2026-09-05 06:29:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:29:16.936983578 +0000 UTC m=+253.620488485" watchObservedRunningTime="2026-09-05 06:29:16.939148108 +0000 UTC m=+253.622653014" Sep 05 06:29:16.943936 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.943883 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e4b646-b3ad-448c-b934-278d1b7b2d59-serving-certs-ca-bundle\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.944082 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.943992 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-telemeter-client-tls\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.944082 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.944020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcwbr\" (UniqueName: \"kubernetes.io/projected/47e4b646-b3ad-448c-b934-278d1b7b2d59-kube-api-access-rcwbr\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.945434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.944246 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-federate-client-tls\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.945434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.944293 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.945434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.944324 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-secret-telemeter-client\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.945434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.944357 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e4b646-b3ad-448c-b934-278d1b7b2d59-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.945434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.944389 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e4b646-b3ad-448c-b934-278d1b7b2d59-metrics-client-ca\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.945796 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.945462 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e4b646-b3ad-448c-b934-278d1b7b2d59-telemeter-trusted-ca-bundle\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.946046 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.946016 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e4b646-b3ad-448c-b934-278d1b7b2d59-serving-certs-ca-bundle\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.946727 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.946663 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e4b646-b3ad-448c-b934-278d1b7b2d59-metrics-client-ca\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.948599 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.948531 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.949057 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.948999 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-telemeter-client-tls\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.949057 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.949017 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-secret-telemeter-client\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.952444 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.949757 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/47e4b646-b3ad-448c-b934-278d1b7b2d59-federate-client-tls\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:16.965562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:16.965507 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcwbr\" (UniqueName: \"kubernetes.io/projected/47e4b646-b3ad-448c-b934-278d1b7b2d59-kube-api-access-rcwbr\") pod \"telemeter-client-b47c7d785-t5d9g\" (UID: \"47e4b646-b3ad-448c-b934-278d1b7b2d59\") " pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:17.073151 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:17.073072 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" Sep 05 06:29:17.403238 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:17.403204 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-b47c7d785-t5d9g"] Sep 05 06:29:17.408498 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:29:17.408067 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e4b646_b3ad_448c_b934_278d1b7b2d59.slice/crio-8d6cc08d23f1309686cc42d37b2c4765041de6b6e4093b02879a7e2daee51cfa WatchSource:0}: Error finding container 8d6cc08d23f1309686cc42d37b2c4765041de6b6e4093b02879a7e2daee51cfa: Status 404 returned error can't find the container with id 8d6cc08d23f1309686cc42d37b2c4765041de6b6e4093b02879a7e2daee51cfa Sep 05 06:29:17.919202 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:17.919161 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m8fn4" event={"ID":"68080680-0551-487c-80ed-b93acb2f68f8","Type":"ContainerStarted","Data":"bb173aa7ddae1d66b71526bdfcdc4081cb77a68aa22d961ed0d1b72c628e363b"} Sep 05 06:29:17.919711 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:17.919233 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m8fn4" event={"ID":"68080680-0551-487c-80ed-b93acb2f68f8","Type":"ContainerStarted","Data":"e48be0e06857793bda52a383061899e11113de2ddeaa630eab9cfd72cc0d5f6c"} Sep 05 06:29:17.920661 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:17.920626 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" event={"ID":"47e4b646-b3ad-448c-b934-278d1b7b2d59","Type":"ContainerStarted","Data":"8d6cc08d23f1309686cc42d37b2c4765041de6b6e4093b02879a7e2daee51cfa"} Sep 05 06:29:17.937620 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:17.937570 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-m8fn4" podStartSLOduration=253.790785626 podStartE2EDuration="4m14.937555493s" totalImagesPullingTime="1.146769867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:29:17.935804575 +0000 UTC m=+254.619309482" watchObservedRunningTime="2026-09-05 06:29:17.937555493 +0000 UTC m=+254.621060399" Sep 05 06:29:19.933593 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:19.933553 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" event={"ID":"47e4b646-b3ad-448c-b934-278d1b7b2d59","Type":"ContainerStarted","Data":"f366b15f3f3fb2d77e30ad43e21a99ad220731ae50475376e3cb39aa448d70aa"} Sep 05 06:29:19.934040 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:19.933619 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" event={"ID":"47e4b646-b3ad-448c-b934-278d1b7b2d59","Type":"ContainerStarted","Data":"0ed94a13ea8ee517cc82783a8c9a93ef9eb787fc64ec3bbc25989a5334cc1419"} Sep 05 06:29:19.934040 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:19.933648 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" event={"ID":"47e4b646-b3ad-448c-b934-278d1b7b2d59","Type":"ContainerStarted","Data":"7727292c731ea994050d3e6420425a02bd7a9a5035a3ec12dbd4c84c1af0ec51"} Sep 05 06:29:19.964326 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:19.964271 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-b47c7d785-t5d9g" podStartSLOduration=2.423351688 podStartE2EDuration="3.964253935s" totalImagesPullingTime="1.540902247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:29:19.962812106 +0000 UTC m=+256.646317036" watchObservedRunningTime="2026-09-05 06:29:19.964253935 +0000 UTC m=+256.647758847" Sep 05 06:29:20.049919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.049888 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c9fbf64c9-66cx6"] Sep 05 06:29:20.052193 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.052162 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.068554 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.068524 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9fbf64c9-66cx6"] Sep 05 06:29:20.174721 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.174683 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-serving-cert\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.174721 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.174733 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-oauth-serving-cert\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.174972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.174762 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd58t\" (UniqueName: \"kubernetes.io/projected/bd716880-56ca-42f9-95d0-d2899e2f37a5-kube-api-access-dd58t\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.174972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.174816 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-trusted-ca-bundle\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.174972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.174847 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-oauth-config\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.174972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.174867 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-config\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.174972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.174891 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-service-ca\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.276079 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.276042 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-serving-cert\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.276079 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.276080 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-oauth-serving-cert\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.276294 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.276102 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd58t\" (UniqueName: \"kubernetes.io/projected/bd716880-56ca-42f9-95d0-d2899e2f37a5-kube-api-access-dd58t\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.276294 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.276228 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-trusted-ca-bundle\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.276294 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.276277 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-oauth-config\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.276442 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.276303 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-config\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.276442 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.276330 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-service-ca\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.276920 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.276894 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-oauth-serving-cert\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.277047 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.276971 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-config\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.277047 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.277019 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-service-ca\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.277155 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.277072 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-trusted-ca-bundle\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.278689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.278667 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-serving-cert\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.278747 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.278733 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-oauth-config\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.286628 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.286605 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd58t\" (UniqueName: \"kubernetes.io/projected/bd716880-56ca-42f9-95d0-d2899e2f37a5-kube-api-access-dd58t\") pod \"console-c9fbf64c9-66cx6\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.362668 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.362632 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:20.633642 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.633616 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9fbf64c9-66cx6"] Sep 05 06:29:20.635745 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:29:20.635721 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd716880_56ca_42f9_95d0_d2899e2f37a5.slice/crio-f23f7abb6792ad9ab2e2dbdde87148277f07fcbfd7021dd13871f85cc8d7627d WatchSource:0}: Error finding container f23f7abb6792ad9ab2e2dbdde87148277f07fcbfd7021dd13871f85cc8d7627d: Status 404 returned error can't find the container with id f23f7abb6792ad9ab2e2dbdde87148277f07fcbfd7021dd13871f85cc8d7627d Sep 05 06:29:20.942587 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.942489 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9fbf64c9-66cx6" event={"ID":"bd716880-56ca-42f9-95d0-d2899e2f37a5","Type":"ContainerStarted","Data":"89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e"} Sep 05 06:29:20.942587 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.942553 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9fbf64c9-66cx6" event={"ID":"bd716880-56ca-42f9-95d0-d2899e2f37a5","Type":"ContainerStarted","Data":"f23f7abb6792ad9ab2e2dbdde87148277f07fcbfd7021dd13871f85cc8d7627d"} Sep 05 06:29:20.964793 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:20.964731 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c9fbf64c9-66cx6" podStartSLOduration=0.964709751 podStartE2EDuration="964.709751ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:29:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:29:20.963299753 +0000 UTC m=+257.646804661" watchObservedRunningTime="2026-09-05 06:29:20.964709751 +0000 UTC m=+257.648214658" Sep 05 06:29:30.363205 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:30.363125 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:30.363205 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:30.363189 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:30.367970 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:30.367945 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:30.993693 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:30.993664 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:29:31.056341 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:31.056309 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7554759b8-cp67b"] Sep 05 06:29:56.077873 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.077811 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7554759b8-cp67b" podUID="9a4d02a8-cd62-47c4-99bc-78dfbce85a56" containerName="console" containerID="cri-o://1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6" gracePeriod=15 Sep 05 06:29:56.316767 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.316738 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:29:56.350155 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.350096 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-trusted-ca-bundle\") pod \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " Sep 05 06:29:56.350267 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.350170 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-oauth-serving-cert\") pod \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " Sep 05 06:29:56.350267 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.350197 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-serving-cert\") pod \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " Sep 05 06:29:56.350267 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.350225 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k6kv\" (UniqueName: \"kubernetes.io/projected/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-kube-api-access-7k6kv\") pod \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " Sep 05 06:29:56.350435 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.350274 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-config\") pod \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " Sep 05 06:29:56.350435 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.350309 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-service-ca\") pod \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " Sep 05 06:29:56.350435 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.350339 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-oauth-config\") pod \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\" (UID: \"9a4d02a8-cd62-47c4-99bc-78dfbce85a56\") " Sep 05 06:29:56.350621 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.350551 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9a4d02a8-cd62-47c4-99bc-78dfbce85a56" (UID: "9a4d02a8-cd62-47c4-99bc-78dfbce85a56"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:29:56.350621 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.350599 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9a4d02a8-cd62-47c4-99bc-78dfbce85a56" (UID: "9a4d02a8-cd62-47c4-99bc-78dfbce85a56"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:29:56.350972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.350940 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-config" (OuterVolumeSpecName: "console-config") pod "9a4d02a8-cd62-47c4-99bc-78dfbce85a56" (UID: "9a4d02a8-cd62-47c4-99bc-78dfbce85a56"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:29:56.351153 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.351128 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-service-ca" (OuterVolumeSpecName: "service-ca") pod "9a4d02a8-cd62-47c4-99bc-78dfbce85a56" (UID: "9a4d02a8-cd62-47c4-99bc-78dfbce85a56"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:29:56.352455 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.352423 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-kube-api-access-7k6kv" (OuterVolumeSpecName: "kube-api-access-7k6kv") pod "9a4d02a8-cd62-47c4-99bc-78dfbce85a56" (UID: "9a4d02a8-cd62-47c4-99bc-78dfbce85a56"). InnerVolumeSpecName "kube-api-access-7k6kv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:29:56.352559 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.352537 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9a4d02a8-cd62-47c4-99bc-78dfbce85a56" (UID: "9a4d02a8-cd62-47c4-99bc-78dfbce85a56"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:29:56.352718 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.352692 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9a4d02a8-cd62-47c4-99bc-78dfbce85a56" (UID: "9a4d02a8-cd62-47c4-99bc-78dfbce85a56"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:29:56.451715 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.451685 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:56.451715 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.451710 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:56.451715 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.451719 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-oauth-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:56.451919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.451727 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-trusted-ca-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:56.451919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.451736 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-oauth-serving-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:56.451919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.451744 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-console-serving-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:56.451919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:56.451752 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k6kv\" (UniqueName: \"kubernetes.io/projected/9a4d02a8-cd62-47c4-99bc-78dfbce85a56-kube-api-access-7k6kv\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:29:57.110381 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:57.110345 2583 generic.go:356] "Generic (PLEG): container finished" podID="9a4d02a8-cd62-47c4-99bc-78dfbce85a56" containerID="1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6" exitCode=0 Sep 05 06:29:57.110800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:57.110389 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7554759b8-cp67b" Sep 05 06:29:57.110800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:57.110407 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7554759b8-cp67b" event={"ID":"9a4d02a8-cd62-47c4-99bc-78dfbce85a56","Type":"ContainerDied","Data":"1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6"} Sep 05 06:29:57.110800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:57.110448 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7554759b8-cp67b" event={"ID":"9a4d02a8-cd62-47c4-99bc-78dfbce85a56","Type":"ContainerDied","Data":"23db67ee644231b9d05f07530dbacf953c014377d36324a050dded8d1fa1093e"} Sep 05 06:29:57.110800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:57.110471 2583 scope.go:117] "RemoveContainer" containerID="1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6" Sep 05 06:29:57.119060 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:57.119044 2583 scope.go:117] "RemoveContainer" containerID="1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6" Sep 05 06:29:57.119318 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:29:57.119294 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6\": container with ID starting with 1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6 not found: ID does not exist" containerID="1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6" Sep 05 06:29:57.119387 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:57.119330 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6"} err="failed to get container status \"1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6\": rpc error: code = NotFound desc = could not find container \"1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6\": container with ID starting with 1ffa9c85f2e17df95d8fc2559c0c49fe93670d56d88d16971e00c7e6bb0d01e6 not found: ID does not exist" Sep 05 06:29:57.131787 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:57.131767 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7554759b8-cp67b"] Sep 05 06:29:57.135484 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:57.135451 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7554759b8-cp67b"] Sep 05 06:29:57.821627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:29:57.821594 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a4d02a8-cd62-47c4-99bc-78dfbce85a56" path="/var/lib/kubelet/pods/9a4d02a8-cd62-47c4-99bc-78dfbce85a56/volumes" Sep 05 06:30:03.729084 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:03.729053 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:30:03.734485 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:03.734450 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:30:03.737421 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:03.737402 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:30:03.741992 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:03.741740 2583 kubelet.go:1648] "Image garbage collection succeeded" Sep 05 06:30:03.743122 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:03.743104 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:30:31.430829 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.430786 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f87dfd46b-fddtz"] Sep 05 06:30:31.433383 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.431151 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4d02a8-cd62-47c4-99bc-78dfbce85a56" containerName="console" Sep 05 06:30:31.433383 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.431165 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4d02a8-cd62-47c4-99bc-78dfbce85a56" containerName="console" Sep 05 06:30:31.433383 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.431234 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a4d02a8-cd62-47c4-99bc-78dfbce85a56" containerName="console" Sep 05 06:30:31.434423 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.434404 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.446568 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.446545 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f87dfd46b-fddtz"] Sep 05 06:30:31.542744 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.542711 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s4w2\" (UniqueName: \"kubernetes.io/projected/d534ebba-5b3a-48c8-976f-9f0942768cbd-kube-api-access-5s4w2\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.542943 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.542757 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-service-ca\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.542943 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.542772 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-trusted-ca-bundle\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.542943 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.542816 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-serving-cert\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.542943 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.542924 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-oauth-serving-cert\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.543114 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.542952 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-config\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.543114 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.542972 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-oauth-config\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.643781 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.643740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s4w2\" (UniqueName: \"kubernetes.io/projected/d534ebba-5b3a-48c8-976f-9f0942768cbd-kube-api-access-5s4w2\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.643781 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.643791 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-service-ca\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.644039 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.643970 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-trusted-ca-bundle\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.644039 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.644007 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-serving-cert\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.644137 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.644051 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-oauth-serving-cert\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.644137 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.644072 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-config\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.644137 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.644093 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-oauth-config\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.644631 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.644601 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-service-ca\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.644836 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.644810 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-oauth-serving-cert\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.644901 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.644835 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-config\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.644901 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.644891 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-trusted-ca-bundle\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.647041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.647014 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-oauth-config\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.647236 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.647212 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-serving-cert\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.652221 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.652196 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s4w2\" (UniqueName: \"kubernetes.io/projected/d534ebba-5b3a-48c8-976f-9f0942768cbd-kube-api-access-5s4w2\") pod \"console-5f87dfd46b-fddtz\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:31.745160 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:31.745129 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:32.017939 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:32.017914 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f87dfd46b-fddtz"] Sep 05 06:30:32.020141 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:30:32.020111 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd534ebba_5b3a_48c8_976f_9f0942768cbd.slice/crio-2ddf50e8e5cfdc9d9960e4d8ac2606cc20059da959ee2219d2850692f8bed845 WatchSource:0}: Error finding container 2ddf50e8e5cfdc9d9960e4d8ac2606cc20059da959ee2219d2850692f8bed845: Status 404 returned error can't find the container with id 2ddf50e8e5cfdc9d9960e4d8ac2606cc20059da959ee2219d2850692f8bed845 Sep 05 06:30:32.257270 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:32.257238 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f87dfd46b-fddtz" event={"ID":"d534ebba-5b3a-48c8-976f-9f0942768cbd","Type":"ContainerStarted","Data":"7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615"} Sep 05 06:30:32.257434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:32.257287 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f87dfd46b-fddtz" event={"ID":"d534ebba-5b3a-48c8-976f-9f0942768cbd","Type":"ContainerStarted","Data":"2ddf50e8e5cfdc9d9960e4d8ac2606cc20059da959ee2219d2850692f8bed845"} Sep 05 06:30:32.278651 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:32.278569 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f87dfd46b-fddtz" podStartSLOduration=1.278551973 podStartE2EDuration="1.278551973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:30:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:30:32.27643913 +0000 UTC m=+328.959944039" watchObservedRunningTime="2026-09-05 06:30:32.278551973 +0000 UTC m=+328.962056879" Sep 05 06:30:41.746183 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:41.746139 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:41.746585 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:41.746208 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:41.750985 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:41.750962 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:42.304854 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:42.304827 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:30:42.354426 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:30:42.354390 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c9fbf64c9-66cx6"] Sep 05 06:31:07.376240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.376138 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c9fbf64c9-66cx6" podUID="bd716880-56ca-42f9-95d0-d2899e2f37a5" containerName="console" containerID="cri-o://89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e" gracePeriod=15 Sep 05 06:31:07.616187 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.616162 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:31:07.743565 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.743541 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-config\") pod \"bd716880-56ca-42f9-95d0-d2899e2f37a5\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " Sep 05 06:31:07.743699 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.743593 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-serving-cert\") pod \"bd716880-56ca-42f9-95d0-d2899e2f37a5\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " Sep 05 06:31:07.743699 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.743647 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-oauth-serving-cert\") pod \"bd716880-56ca-42f9-95d0-d2899e2f37a5\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " Sep 05 06:31:07.743804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.743699 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-trusted-ca-bundle\") pod \"bd716880-56ca-42f9-95d0-d2899e2f37a5\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " Sep 05 06:31:07.743804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.743721 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-service-ca\") pod \"bd716880-56ca-42f9-95d0-d2899e2f37a5\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " Sep 05 06:31:07.743804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.743748 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd58t\" (UniqueName: \"kubernetes.io/projected/bd716880-56ca-42f9-95d0-d2899e2f37a5-kube-api-access-dd58t\") pod \"bd716880-56ca-42f9-95d0-d2899e2f37a5\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " Sep 05 06:31:07.743804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.743770 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-oauth-config\") pod \"bd716880-56ca-42f9-95d0-d2899e2f37a5\" (UID: \"bd716880-56ca-42f9-95d0-d2899e2f37a5\") " Sep 05 06:31:07.743995 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.743902 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-config" (OuterVolumeSpecName: "console-config") pod "bd716880-56ca-42f9-95d0-d2899e2f37a5" (UID: "bd716880-56ca-42f9-95d0-d2899e2f37a5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:31:07.744054 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.744030 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:31:07.744200 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.744165 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-service-ca" (OuterVolumeSpecName: "service-ca") pod "bd716880-56ca-42f9-95d0-d2899e2f37a5" (UID: "bd716880-56ca-42f9-95d0-d2899e2f37a5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:31:07.744200 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.744179 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bd716880-56ca-42f9-95d0-d2899e2f37a5" (UID: "bd716880-56ca-42f9-95d0-d2899e2f37a5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:31:07.744200 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.744181 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bd716880-56ca-42f9-95d0-d2899e2f37a5" (UID: "bd716880-56ca-42f9-95d0-d2899e2f37a5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:31:07.745776 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.745754 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bd716880-56ca-42f9-95d0-d2899e2f37a5" (UID: "bd716880-56ca-42f9-95d0-d2899e2f37a5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:31:07.745882 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.745867 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bd716880-56ca-42f9-95d0-d2899e2f37a5" (UID: "bd716880-56ca-42f9-95d0-d2899e2f37a5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:31:07.745882 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.745866 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd716880-56ca-42f9-95d0-d2899e2f37a5-kube-api-access-dd58t" (OuterVolumeSpecName: "kube-api-access-dd58t") pod "bd716880-56ca-42f9-95d0-d2899e2f37a5" (UID: "bd716880-56ca-42f9-95d0-d2899e2f37a5"). InnerVolumeSpecName "kube-api-access-dd58t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:31:07.844925 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.844903 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-serving-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:31:07.844925 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.844924 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-oauth-serving-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:31:07.845049 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.844934 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-trusted-ca-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:31:07.845049 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.844943 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bd716880-56ca-42f9-95d0-d2899e2f37a5-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:31:07.845049 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.844951 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd58t\" (UniqueName: \"kubernetes.io/projected/bd716880-56ca-42f9-95d0-d2899e2f37a5-kube-api-access-dd58t\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:31:07.845049 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:07.844961 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bd716880-56ca-42f9-95d0-d2899e2f37a5-console-oauth-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:31:08.426469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:08.426431 2583 generic.go:356] "Generic (PLEG): container finished" podID="bd716880-56ca-42f9-95d0-d2899e2f37a5" containerID="89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e" exitCode=0 Sep 05 06:31:08.426469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:08.426486 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9fbf64c9-66cx6" Sep 05 06:31:08.426922 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:08.426522 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9fbf64c9-66cx6" event={"ID":"bd716880-56ca-42f9-95d0-d2899e2f37a5","Type":"ContainerDied","Data":"89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e"} Sep 05 06:31:08.426922 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:08.426575 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9fbf64c9-66cx6" event={"ID":"bd716880-56ca-42f9-95d0-d2899e2f37a5","Type":"ContainerDied","Data":"f23f7abb6792ad9ab2e2dbdde87148277f07fcbfd7021dd13871f85cc8d7627d"} Sep 05 06:31:08.426922 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:08.426597 2583 scope.go:117] "RemoveContainer" containerID="89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e" Sep 05 06:31:08.435178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:08.435163 2583 scope.go:117] "RemoveContainer" containerID="89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e" Sep 05 06:31:08.435437 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:31:08.435416 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e\": container with ID starting with 89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e not found: ID does not exist" containerID="89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e" Sep 05 06:31:08.435502 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:08.435445 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e"} err="failed to get container status \"89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e\": rpc error: code = NotFound desc = could not find container \"89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e\": container with ID starting with 89f57bbd1fdcb2084eb88317beacff5f10773033e428ea61ca8edc4f9129102e not found: ID does not exist" Sep 05 06:31:08.444605 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:08.444586 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c9fbf64c9-66cx6"] Sep 05 06:31:08.449066 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:08.449047 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c9fbf64c9-66cx6"] Sep 05 06:31:09.822067 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:31:09.822030 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd716880-56ca-42f9-95d0-d2899e2f37a5" path="/var/lib/kubelet/pods/bd716880-56ca-42f9-95d0-d2899e2f37a5/volumes" Sep 05 06:32:00.368646 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.368610 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk"] Sep 05 06:32:00.369080 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.368960 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd716880-56ca-42f9-95d0-d2899e2f37a5" containerName="console" Sep 05 06:32:00.369080 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.368974 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd716880-56ca-42f9-95d0-d2899e2f37a5" containerName="console" Sep 05 06:32:00.369080 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.369056 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd716880-56ca-42f9-95d0-d2899e2f37a5" containerName="console" Sep 05 06:32:00.372170 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.372154 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.374746 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.374727 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-plzhh\"" Sep 05 06:32:00.374830 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.374798 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 05 06:32:00.375664 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.375647 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 05 06:32:00.382030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.382006 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk"] Sep 05 06:32:00.469178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.469147 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.469178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.469180 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjnbr\" (UniqueName: \"kubernetes.io/projected/347de643-cff7-44b2-965d-bbdcc299fd64-kube-api-access-zjnbr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.469395 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.469245 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.570359 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.570326 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.570359 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.570362 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjnbr\" (UniqueName: \"kubernetes.io/projected/347de643-cff7-44b2-965d-bbdcc299fd64-kube-api-access-zjnbr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.570555 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.570396 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.570721 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.570700 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.570758 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.570718 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.579380 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.579354 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjnbr\" (UniqueName: \"kubernetes.io/projected/347de643-cff7-44b2-965d-bbdcc299fd64-kube-api-access-zjnbr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.681632 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.681551 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:00.956033 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.954660 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk"] Sep 05 06:32:00.958212 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:32:00.958182 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347de643_cff7_44b2_965d_bbdcc299fd64.slice/crio-dd9ced2e731c55d04bd58da4d3ffe7ee62df89c1bd3bb5ca82289fbde503ae1d WatchSource:0}: Error finding container dd9ced2e731c55d04bd58da4d3ffe7ee62df89c1bd3bb5ca82289fbde503ae1d: Status 404 returned error can't find the container with id dd9ced2e731c55d04bd58da4d3ffe7ee62df89c1bd3bb5ca82289fbde503ae1d Sep 05 06:32:00.960217 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:00.960202 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 05 06:32:01.677797 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:01.677763 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" event={"ID":"347de643-cff7-44b2-965d-bbdcc299fd64","Type":"ContainerStarted","Data":"dd9ced2e731c55d04bd58da4d3ffe7ee62df89c1bd3bb5ca82289fbde503ae1d"} Sep 05 06:32:06.702958 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:06.702924 2583 generic.go:356] "Generic (PLEG): container finished" podID="347de643-cff7-44b2-965d-bbdcc299fd64" containerID="f13cb1ee14f9b9b60978429e0f8f2f2af1db6e1c291f7035dd9b643991e2198e" exitCode=0 Sep 05 06:32:06.703339 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:06.703010 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" event={"ID":"347de643-cff7-44b2-965d-bbdcc299fd64","Type":"ContainerDied","Data":"f13cb1ee14f9b9b60978429e0f8f2f2af1db6e1c291f7035dd9b643991e2198e"} Sep 05 06:32:10.722494 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:10.722443 2583 generic.go:356] "Generic (PLEG): container finished" podID="347de643-cff7-44b2-965d-bbdcc299fd64" containerID="b4f8be4ebc084667103afb493be1c425451cf20c06d3786052966e6f579b7ac1" exitCode=0 Sep 05 06:32:10.722822 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:10.722513 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" event={"ID":"347de643-cff7-44b2-965d-bbdcc299fd64","Type":"ContainerDied","Data":"b4f8be4ebc084667103afb493be1c425451cf20c06d3786052966e6f579b7ac1"} Sep 05 06:32:17.759317 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:17.759275 2583 generic.go:356] "Generic (PLEG): container finished" podID="347de643-cff7-44b2-965d-bbdcc299fd64" containerID="2769920c379523604e433faa56cfe7030fc92f45a3b3d7618997b3d8bd8e5093" exitCode=0 Sep 05 06:32:17.759701 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:17.759367 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" event={"ID":"347de643-cff7-44b2-965d-bbdcc299fd64","Type":"ContainerDied","Data":"2769920c379523604e433faa56cfe7030fc92f45a3b3d7618997b3d8bd8e5093"} Sep 05 06:32:18.896714 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:18.896687 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:19.036933 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.036858 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-util\") pod \"347de643-cff7-44b2-965d-bbdcc299fd64\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " Sep 05 06:32:19.036933 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.036902 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjnbr\" (UniqueName: \"kubernetes.io/projected/347de643-cff7-44b2-965d-bbdcc299fd64-kube-api-access-zjnbr\") pod \"347de643-cff7-44b2-965d-bbdcc299fd64\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " Sep 05 06:32:19.037112 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.036967 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-bundle\") pod \"347de643-cff7-44b2-965d-bbdcc299fd64\" (UID: \"347de643-cff7-44b2-965d-bbdcc299fd64\") " Sep 05 06:32:19.037688 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.037656 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-bundle" (OuterVolumeSpecName: "bundle") pod "347de643-cff7-44b2-965d-bbdcc299fd64" (UID: "347de643-cff7-44b2-965d-bbdcc299fd64"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:32:19.039142 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.039114 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347de643-cff7-44b2-965d-bbdcc299fd64-kube-api-access-zjnbr" (OuterVolumeSpecName: "kube-api-access-zjnbr") pod "347de643-cff7-44b2-965d-bbdcc299fd64" (UID: "347de643-cff7-44b2-965d-bbdcc299fd64"). InnerVolumeSpecName "kube-api-access-zjnbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:32:19.040837 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.040819 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-util" (OuterVolumeSpecName: "util") pod "347de643-cff7-44b2-965d-bbdcc299fd64" (UID: "347de643-cff7-44b2-965d-bbdcc299fd64"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:32:19.138421 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.138395 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjnbr\" (UniqueName: \"kubernetes.io/projected/347de643-cff7-44b2-965d-bbdcc299fd64-kube-api-access-zjnbr\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:32:19.138421 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.138419 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:32:19.138596 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.138428 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347de643-cff7-44b2-965d-bbdcc299fd64-util\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:32:19.776211 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.776177 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" event={"ID":"347de643-cff7-44b2-965d-bbdcc299fd64","Type":"ContainerDied","Data":"dd9ced2e731c55d04bd58da4d3ffe7ee62df89c1bd3bb5ca82289fbde503ae1d"} Sep 05 06:32:19.776211 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.776203 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405mdvk" Sep 05 06:32:19.776410 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:19.776249 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd9ced2e731c55d04bd58da4d3ffe7ee62df89c1bd3bb5ca82289fbde503ae1d" Sep 05 06:32:27.314758 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.314682 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg"] Sep 05 06:32:27.315171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.315046 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347de643-cff7-44b2-965d-bbdcc299fd64" containerName="util" Sep 05 06:32:27.315171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.315059 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="347de643-cff7-44b2-965d-bbdcc299fd64" containerName="util" Sep 05 06:32:27.315171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.315068 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347de643-cff7-44b2-965d-bbdcc299fd64" containerName="pull" Sep 05 06:32:27.315171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.315074 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="347de643-cff7-44b2-965d-bbdcc299fd64" containerName="pull" Sep 05 06:32:27.315171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.315100 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347de643-cff7-44b2-965d-bbdcc299fd64" containerName="extract" Sep 05 06:32:27.315171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.315109 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="347de643-cff7-44b2-965d-bbdcc299fd64" containerName="extract" Sep 05 06:32:27.315171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.315172 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="347de643-cff7-44b2-965d-bbdcc299fd64" containerName="extract" Sep 05 06:32:27.324722 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.324701 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.327738 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.327708 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 05 06:32:27.328663 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.328646 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-plzhh\"" Sep 05 06:32:27.328784 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.328655 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 05 06:32:27.335824 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.335801 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg"] Sep 05 06:32:27.406172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.406139 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrxvv\" (UniqueName: \"kubernetes.io/projected/46447b73-48d0-4bcd-b248-c859e6d520c0-kube-api-access-qrxvv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.406351 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.406199 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.406351 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.406242 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.507152 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.507112 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrxvv\" (UniqueName: \"kubernetes.io/projected/46447b73-48d0-4bcd-b248-c859e6d520c0-kube-api-access-qrxvv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.507329 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.507187 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.507329 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.507228 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.507573 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.507552 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.507667 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.507641 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.519031 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.519003 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrxvv\" (UniqueName: \"kubernetes.io/projected/46447b73-48d0-4bcd-b248-c859e6d520c0-kube-api-access-qrxvv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.634044 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.633968 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:27.917642 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:27.917619 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg"] Sep 05 06:32:27.919500 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:32:27.919455 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46447b73_48d0_4bcd_b248_c859e6d520c0.slice/crio-4bb07d6859d8163c5121097f0f1e7571d1ebd4ee84ecc4975c76c356bb65cb36 WatchSource:0}: Error finding container 4bb07d6859d8163c5121097f0f1e7571d1ebd4ee84ecc4975c76c356bb65cb36: Status 404 returned error can't find the container with id 4bb07d6859d8163c5121097f0f1e7571d1ebd4ee84ecc4975c76c356bb65cb36 Sep 05 06:32:28.400218 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.400181 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qqjd7"] Sep 05 06:32:28.402545 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.402523 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" Sep 05 06:32:28.405907 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.405891 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 05 06:32:28.405997 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.405903 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 05 06:32:28.406709 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.406693 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-r8qrh\"" Sep 05 06:32:28.414091 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.414066 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e10d61a-f345-441b-9fc6-650e8d7fd405-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qqjd7\" (UID: \"7e10d61a-f345-441b-9fc6-650e8d7fd405\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" Sep 05 06:32:28.414188 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.414103 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcks7\" (UniqueName: \"kubernetes.io/projected/7e10d61a-f345-441b-9fc6-650e8d7fd405-kube-api-access-mcks7\") pod \"cert-manager-webhook-6486ffd997-qqjd7\" (UID: \"7e10d61a-f345-441b-9fc6-650e8d7fd405\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" Sep 05 06:32:28.415942 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.415923 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qqjd7"] Sep 05 06:32:28.514809 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.514778 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e10d61a-f345-441b-9fc6-650e8d7fd405-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qqjd7\" (UID: \"7e10d61a-f345-441b-9fc6-650e8d7fd405\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" Sep 05 06:32:28.514975 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.514816 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcks7\" (UniqueName: \"kubernetes.io/projected/7e10d61a-f345-441b-9fc6-650e8d7fd405-kube-api-access-mcks7\") pod \"cert-manager-webhook-6486ffd997-qqjd7\" (UID: \"7e10d61a-f345-441b-9fc6-650e8d7fd405\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" Sep 05 06:32:28.524351 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.524325 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e10d61a-f345-441b-9fc6-650e8d7fd405-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qqjd7\" (UID: \"7e10d61a-f345-441b-9fc6-650e8d7fd405\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" Sep 05 06:32:28.524613 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.524595 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcks7\" (UniqueName: \"kubernetes.io/projected/7e10d61a-f345-441b-9fc6-650e8d7fd405-kube-api-access-mcks7\") pod \"cert-manager-webhook-6486ffd997-qqjd7\" (UID: \"7e10d61a-f345-441b-9fc6-650e8d7fd405\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" Sep 05 06:32:28.725606 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.725526 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" Sep 05 06:32:28.818040 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.818005 2583 generic.go:356] "Generic (PLEG): container finished" podID="46447b73-48d0-4bcd-b248-c859e6d520c0" containerID="b1c05f7cec753c7be7c0e6296833c59cbb8487db5d57bd88b9559cfbdf75c16e" exitCode=0 Sep 05 06:32:28.818171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.818055 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" event={"ID":"46447b73-48d0-4bcd-b248-c859e6d520c0","Type":"ContainerDied","Data":"b1c05f7cec753c7be7c0e6296833c59cbb8487db5d57bd88b9559cfbdf75c16e"} Sep 05 06:32:28.818171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.818088 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" event={"ID":"46447b73-48d0-4bcd-b248-c859e6d520c0","Type":"ContainerStarted","Data":"4bb07d6859d8163c5121097f0f1e7571d1ebd4ee84ecc4975c76c356bb65cb36"} Sep 05 06:32:28.995102 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:28.995074 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qqjd7"] Sep 05 06:32:28.998618 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:32:28.998595 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e10d61a_f345_441b_9fc6_650e8d7fd405.slice/crio-1e63e53e7a49a5f9100d8e13857bb6ae6751385eb2695f4ee83e53de13aae628 WatchSource:0}: Error finding container 1e63e53e7a49a5f9100d8e13857bb6ae6751385eb2695f4ee83e53de13aae628: Status 404 returned error can't find the container with id 1e63e53e7a49a5f9100d8e13857bb6ae6751385eb2695f4ee83e53de13aae628 Sep 05 06:32:29.827691 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:29.827633 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" event={"ID":"7e10d61a-f345-441b-9fc6-650e8d7fd405","Type":"ContainerStarted","Data":"1e63e53e7a49a5f9100d8e13857bb6ae6751385eb2695f4ee83e53de13aae628"} Sep 05 06:32:29.952464 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:29.952435 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-85l2n"] Sep 05 06:32:29.956333 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:29.956312 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" Sep 05 06:32:29.959001 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:29.958976 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qxqmb\"" Sep 05 06:32:29.969775 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:29.969732 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-85l2n"] Sep 05 06:32:30.026007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:30.025968 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l98hl\" (UniqueName: \"kubernetes.io/projected/d13cb890-e388-45fc-babb-c61fa9f38370-kube-api-access-l98hl\") pod \"cert-manager-cainjector-559cdd8c87-85l2n\" (UID: \"d13cb890-e388-45fc-babb-c61fa9f38370\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" Sep 05 06:32:30.026188 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:30.026058 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d13cb890-e388-45fc-babb-c61fa9f38370-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-85l2n\" (UID: \"d13cb890-e388-45fc-babb-c61fa9f38370\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" Sep 05 06:32:30.126968 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:30.126876 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l98hl\" (UniqueName: \"kubernetes.io/projected/d13cb890-e388-45fc-babb-c61fa9f38370-kube-api-access-l98hl\") pod \"cert-manager-cainjector-559cdd8c87-85l2n\" (UID: \"d13cb890-e388-45fc-babb-c61fa9f38370\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" Sep 05 06:32:30.127127 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:30.126990 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d13cb890-e388-45fc-babb-c61fa9f38370-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-85l2n\" (UID: \"d13cb890-e388-45fc-babb-c61fa9f38370\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" Sep 05 06:32:30.136385 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:30.136333 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d13cb890-e388-45fc-babb-c61fa9f38370-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-85l2n\" (UID: \"d13cb890-e388-45fc-babb-c61fa9f38370\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" Sep 05 06:32:30.136537 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:30.136503 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l98hl\" (UniqueName: \"kubernetes.io/projected/d13cb890-e388-45fc-babb-c61fa9f38370-kube-api-access-l98hl\") pod \"cert-manager-cainjector-559cdd8c87-85l2n\" (UID: \"d13cb890-e388-45fc-babb-c61fa9f38370\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" Sep 05 06:32:30.268847 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:30.268802 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" Sep 05 06:32:31.029780 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:31.029756 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-85l2n"] Sep 05 06:32:31.843972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:31.843931 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" event={"ID":"46447b73-48d0-4bcd-b248-c859e6d520c0","Type":"ContainerStarted","Data":"441065bf4fe47d2e3fd9ca3345d7c8e2fdd55e7c20429b246513292ed98052d7"} Sep 05 06:32:31.880305 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:32:31.880268 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd13cb890_e388_45fc_babb_c61fa9f38370.slice/crio-3b824deb40c2651c7d13a234c823f32fe0c832ca490508efc85a99902ddcc925 WatchSource:0}: Error finding container 3b824deb40c2651c7d13a234c823f32fe0c832ca490508efc85a99902ddcc925: Status 404 returned error can't find the container with id 3b824deb40c2651c7d13a234c823f32fe0c832ca490508efc85a99902ddcc925 Sep 05 06:32:32.852679 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:32.852639 2583 generic.go:356] "Generic (PLEG): container finished" podID="46447b73-48d0-4bcd-b248-c859e6d520c0" containerID="441065bf4fe47d2e3fd9ca3345d7c8e2fdd55e7c20429b246513292ed98052d7" exitCode=0 Sep 05 06:32:32.853055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:32.852719 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" event={"ID":"46447b73-48d0-4bcd-b248-c859e6d520c0","Type":"ContainerDied","Data":"441065bf4fe47d2e3fd9ca3345d7c8e2fdd55e7c20429b246513292ed98052d7"} Sep 05 06:32:32.854384 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:32.854361 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" event={"ID":"d13cb890-e388-45fc-babb-c61fa9f38370","Type":"ContainerStarted","Data":"db5743a08f6399b43cf8607401fb31fac113968c20a1f32b64919940fe3eaa6c"} Sep 05 06:32:32.854508 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:32.854394 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" event={"ID":"d13cb890-e388-45fc-babb-c61fa9f38370","Type":"ContainerStarted","Data":"3b824deb40c2651c7d13a234c823f32fe0c832ca490508efc85a99902ddcc925"} Sep 05 06:32:32.855920 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:32.855898 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" event={"ID":"7e10d61a-f345-441b-9fc6-650e8d7fd405","Type":"ContainerStarted","Data":"5833a8f38ae5d07f103f1feaa6ba7610ac18fd82180886b3c747d2de3ba7afbe"} Sep 05 06:32:32.856083 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:32.856035 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" Sep 05 06:32:32.921177 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:32.921133 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" podStartSLOduration=1.985045487 podStartE2EDuration="4.921117006s" totalImagesPullingTime="2.936071519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:32:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:32:32.92011638 +0000 UTC m=+449.603621287" watchObservedRunningTime="2026-09-05 06:32:32.921117006 +0000 UTC m=+449.604621912" Sep 05 06:32:32.948364 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:32.948315 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-85l2n" podStartSLOduration=3.537279318 podStartE2EDuration="3.948297606s" totalImagesPullingTime="411.018288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:32:32.947853413 +0000 UTC m=+449.631358320" watchObservedRunningTime="2026-09-05 06:32:32.948297606 +0000 UTC m=+449.631802513" Sep 05 06:32:33.862849 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:33.862819 2583 generic.go:356] "Generic (PLEG): container finished" podID="46447b73-48d0-4bcd-b248-c859e6d520c0" containerID="1f95da4047d5b26963ee235f5b3781cb31c4f4ce4741ae796ae64acca4627272" exitCode=0 Sep 05 06:32:33.863277 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:33.862906 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" event={"ID":"46447b73-48d0-4bcd-b248-c859e6d520c0","Type":"ContainerDied","Data":"1f95da4047d5b26963ee235f5b3781cb31c4f4ce4741ae796ae64acca4627272"} Sep 05 06:32:34.993139 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:34.993115 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:35.069269 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.069234 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-util\") pod \"46447b73-48d0-4bcd-b248-c859e6d520c0\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " Sep 05 06:32:35.069452 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.069330 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-bundle\") pod \"46447b73-48d0-4bcd-b248-c859e6d520c0\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " Sep 05 06:32:35.069452 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.069360 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrxvv\" (UniqueName: \"kubernetes.io/projected/46447b73-48d0-4bcd-b248-c859e6d520c0-kube-api-access-qrxvv\") pod \"46447b73-48d0-4bcd-b248-c859e6d520c0\" (UID: \"46447b73-48d0-4bcd-b248-c859e6d520c0\") " Sep 05 06:32:35.069750 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.069725 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-bundle" (OuterVolumeSpecName: "bundle") pod "46447b73-48d0-4bcd-b248-c859e6d520c0" (UID: "46447b73-48d0-4bcd-b248-c859e6d520c0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:32:35.071448 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.071409 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46447b73-48d0-4bcd-b248-c859e6d520c0-kube-api-access-qrxvv" (OuterVolumeSpecName: "kube-api-access-qrxvv") pod "46447b73-48d0-4bcd-b248-c859e6d520c0" (UID: "46447b73-48d0-4bcd-b248-c859e6d520c0"). InnerVolumeSpecName "kube-api-access-qrxvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:32:35.073635 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.073616 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-util" (OuterVolumeSpecName: "util") pod "46447b73-48d0-4bcd-b248-c859e6d520c0" (UID: "46447b73-48d0-4bcd-b248-c859e6d520c0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:32:35.170006 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.169931 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:32:35.170006 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.169958 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrxvv\" (UniqueName: \"kubernetes.io/projected/46447b73-48d0-4bcd-b248-c859e6d520c0-kube-api-access-qrxvv\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:32:35.170006 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.169967 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46447b73-48d0-4bcd-b248-c859e6d520c0-util\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:32:35.874340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.874313 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" Sep 05 06:32:35.874340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.874323 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv8nmg" event={"ID":"46447b73-48d0-4bcd-b248-c859e6d520c0","Type":"ContainerDied","Data":"4bb07d6859d8163c5121097f0f1e7571d1ebd4ee84ecc4975c76c356bb65cb36"} Sep 05 06:32:35.874564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:35.874369 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bb07d6859d8163c5121097f0f1e7571d1ebd4ee84ecc4975c76c356bb65cb36" Sep 05 06:32:37.709507 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.709457 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-q5gh9"] Sep 05 06:32:37.709892 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.709856 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46447b73-48d0-4bcd-b248-c859e6d520c0" containerName="util" Sep 05 06:32:37.709892 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.709869 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="46447b73-48d0-4bcd-b248-c859e6d520c0" containerName="util" Sep 05 06:32:37.709892 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.709891 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46447b73-48d0-4bcd-b248-c859e6d520c0" containerName="extract" Sep 05 06:32:37.710007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.709897 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="46447b73-48d0-4bcd-b248-c859e6d520c0" containerName="extract" Sep 05 06:32:37.710007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.709906 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46447b73-48d0-4bcd-b248-c859e6d520c0" containerName="pull" Sep 05 06:32:37.710007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.709914 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="46447b73-48d0-4bcd-b248-c859e6d520c0" containerName="pull" Sep 05 06:32:37.710007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.709965 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="46447b73-48d0-4bcd-b248-c859e6d520c0" containerName="extract" Sep 05 06:32:37.713905 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.713887 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-q5gh9" Sep 05 06:32:37.716336 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.716319 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wcxd9\"" Sep 05 06:32:37.723128 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.723106 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-q5gh9"] Sep 05 06:32:37.791263 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.791236 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23d4d4cb-350d-4cec-a847-997942b66482-bound-sa-token\") pod \"cert-manager-55b4d954d5-q5gh9\" (UID: \"23d4d4cb-350d-4cec-a847-997942b66482\") " pod="cert-manager/cert-manager-55b4d954d5-q5gh9" Sep 05 06:32:37.791412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.791297 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl7gs\" (UniqueName: \"kubernetes.io/projected/23d4d4cb-350d-4cec-a847-997942b66482-kube-api-access-hl7gs\") pod \"cert-manager-55b4d954d5-q5gh9\" (UID: \"23d4d4cb-350d-4cec-a847-997942b66482\") " pod="cert-manager/cert-manager-55b4d954d5-q5gh9" Sep 05 06:32:37.891941 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.891913 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23d4d4cb-350d-4cec-a847-997942b66482-bound-sa-token\") pod \"cert-manager-55b4d954d5-q5gh9\" (UID: \"23d4d4cb-350d-4cec-a847-997942b66482\") " pod="cert-manager/cert-manager-55b4d954d5-q5gh9" Sep 05 06:32:37.892073 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.891988 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl7gs\" (UniqueName: \"kubernetes.io/projected/23d4d4cb-350d-4cec-a847-997942b66482-kube-api-access-hl7gs\") pod \"cert-manager-55b4d954d5-q5gh9\" (UID: \"23d4d4cb-350d-4cec-a847-997942b66482\") " pod="cert-manager/cert-manager-55b4d954d5-q5gh9" Sep 05 06:32:37.901317 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.901283 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23d4d4cb-350d-4cec-a847-997942b66482-bound-sa-token\") pod \"cert-manager-55b4d954d5-q5gh9\" (UID: \"23d4d4cb-350d-4cec-a847-997942b66482\") " pod="cert-manager/cert-manager-55b4d954d5-q5gh9" Sep 05 06:32:37.901423 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:37.901398 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl7gs\" (UniqueName: \"kubernetes.io/projected/23d4d4cb-350d-4cec-a847-997942b66482-kube-api-access-hl7gs\") pod \"cert-manager-55b4d954d5-q5gh9\" (UID: \"23d4d4cb-350d-4cec-a847-997942b66482\") " pod="cert-manager/cert-manager-55b4d954d5-q5gh9" Sep 05 06:32:38.023470 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:38.023444 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-q5gh9" Sep 05 06:32:38.292913 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:38.292841 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-q5gh9"] Sep 05 06:32:38.297240 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:32:38.297205 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23d4d4cb_350d_4cec_a847_997942b66482.slice/crio-828571be6dd6688bba07b9164f62259d8136580d4e7ace0eb0374a6577d17772 WatchSource:0}: Error finding container 828571be6dd6688bba07b9164f62259d8136580d4e7ace0eb0374a6577d17772: Status 404 returned error can't find the container with id 828571be6dd6688bba07b9164f62259d8136580d4e7ace0eb0374a6577d17772 Sep 05 06:32:38.865707 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:38.865674 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qqjd7" Sep 05 06:32:38.891826 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:38.891792 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-q5gh9" event={"ID":"23d4d4cb-350d-4cec-a847-997942b66482","Type":"ContainerStarted","Data":"ce8dd620c866ef73abc34c6de9e3e8c36d2e91940686767b8c76882d8b6496b6"} Sep 05 06:32:38.892003 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:38.891839 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-q5gh9" event={"ID":"23d4d4cb-350d-4cec-a847-997942b66482","Type":"ContainerStarted","Data":"828571be6dd6688bba07b9164f62259d8136580d4e7ace0eb0374a6577d17772"} Sep 05 06:32:38.908542 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:38.908469 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-q5gh9" podStartSLOduration=1.9084536600000002 podStartE2EDuration="1.90845366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:32:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:32:38.907627045 +0000 UTC m=+455.591131953" watchObservedRunningTime="2026-09-05 06:32:38.90845366 +0000 UTC m=+455.591958567" Sep 05 06:32:47.869261 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:47.869224 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4"] Sep 05 06:32:47.873282 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:47.873256 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:47.875784 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:47.875762 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 05 06:32:47.876624 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:47.876604 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-plzhh\"" Sep 05 06:32:47.876741 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:47.876655 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 05 06:32:47.881761 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:47.881720 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4"] Sep 05 06:32:47.974370 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:47.974342 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:47.974562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:47.974402 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:47.974562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:47.974431 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9nlg\" (UniqueName: \"kubernetes.io/projected/82bf35cd-11ab-4cd7-a90c-980d2630ca59-kube-api-access-j9nlg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:48.075922 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.075891 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:48.076087 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.075958 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:48.076087 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.075982 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9nlg\" (UniqueName: \"kubernetes.io/projected/82bf35cd-11ab-4cd7-a90c-980d2630ca59-kube-api-access-j9nlg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:48.076275 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.076257 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:48.076313 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.076280 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:48.086162 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.086134 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9nlg\" (UniqueName: \"kubernetes.io/projected/82bf35cd-11ab-4cd7-a90c-980d2630ca59-kube-api-access-j9nlg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:48.184008 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.183916 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:48.452732 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.452708 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4"] Sep 05 06:32:48.454603 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:32:48.454579 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82bf35cd_11ab_4cd7_a90c_980d2630ca59.slice/crio-1b75788d31c2b413c21be7642abf31e6da25b3895f319338204fd95446c71677 WatchSource:0}: Error finding container 1b75788d31c2b413c21be7642abf31e6da25b3895f319338204fd95446c71677: Status 404 returned error can't find the container with id 1b75788d31c2b413c21be7642abf31e6da25b3895f319338204fd95446c71677 Sep 05 06:32:48.943706 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.943672 2583 generic.go:356] "Generic (PLEG): container finished" podID="82bf35cd-11ab-4cd7-a90c-980d2630ca59" containerID="17645bf76f5ed2ffa19bb6b4c8a3d14b9d469c088875097312de12458b9de96f" exitCode=0 Sep 05 06:32:48.944075 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.943763 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" event={"ID":"82bf35cd-11ab-4cd7-a90c-980d2630ca59","Type":"ContainerDied","Data":"17645bf76f5ed2ffa19bb6b4c8a3d14b9d469c088875097312de12458b9de96f"} Sep 05 06:32:48.944075 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:48.943816 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" event={"ID":"82bf35cd-11ab-4cd7-a90c-980d2630ca59","Type":"ContainerStarted","Data":"1b75788d31c2b413c21be7642abf31e6da25b3895f319338204fd95446c71677"} Sep 05 06:32:49.950592 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:49.950557 2583 generic.go:356] "Generic (PLEG): container finished" podID="82bf35cd-11ab-4cd7-a90c-980d2630ca59" containerID="26c4624e72800eb6cb2b596f7ca40aef77021d04685f7dabe324ce106964b06d" exitCode=0 Sep 05 06:32:49.951003 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:49.950643 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" event={"ID":"82bf35cd-11ab-4cd7-a90c-980d2630ca59","Type":"ContainerDied","Data":"26c4624e72800eb6cb2b596f7ca40aef77021d04685f7dabe324ce106964b06d"} Sep 05 06:32:50.957542 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:50.957505 2583 generic.go:356] "Generic (PLEG): container finished" podID="82bf35cd-11ab-4cd7-a90c-980d2630ca59" containerID="aa17b4793b102b0585b8bd4161833ce9404016bc013ab98a138dae037cd17ee3" exitCode=0 Sep 05 06:32:50.957903 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:50.957565 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" event={"ID":"82bf35cd-11ab-4cd7-a90c-980d2630ca59","Type":"ContainerDied","Data":"aa17b4793b102b0585b8bd4161833ce9404016bc013ab98a138dae037cd17ee3"} Sep 05 06:32:52.084625 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.084598 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:52.209414 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.209384 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-util\") pod \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " Sep 05 06:32:52.209592 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.209431 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9nlg\" (UniqueName: \"kubernetes.io/projected/82bf35cd-11ab-4cd7-a90c-980d2630ca59-kube-api-access-j9nlg\") pod \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " Sep 05 06:32:52.209592 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.209456 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-bundle\") pod \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\" (UID: \"82bf35cd-11ab-4cd7-a90c-980d2630ca59\") " Sep 05 06:32:52.210169 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.210145 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-bundle" (OuterVolumeSpecName: "bundle") pod "82bf35cd-11ab-4cd7-a90c-980d2630ca59" (UID: "82bf35cd-11ab-4cd7-a90c-980d2630ca59"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:32:52.211429 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.211401 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82bf35cd-11ab-4cd7-a90c-980d2630ca59-kube-api-access-j9nlg" (OuterVolumeSpecName: "kube-api-access-j9nlg") pod "82bf35cd-11ab-4cd7-a90c-980d2630ca59" (UID: "82bf35cd-11ab-4cd7-a90c-980d2630ca59"). InnerVolumeSpecName "kube-api-access-j9nlg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:32:52.215120 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.215062 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-util" (OuterVolumeSpecName: "util") pod "82bf35cd-11ab-4cd7-a90c-980d2630ca59" (UID: "82bf35cd-11ab-4cd7-a90c-980d2630ca59"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:32:52.310197 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.310167 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-util\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:32:52.310197 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.310192 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9nlg\" (UniqueName: \"kubernetes.io/projected/82bf35cd-11ab-4cd7-a90c-980d2630ca59-kube-api-access-j9nlg\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:32:52.310197 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.310203 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82bf35cd-11ab-4cd7-a90c-980d2630ca59-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:32:52.968908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.968877 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" event={"ID":"82bf35cd-11ab-4cd7-a90c-980d2630ca59","Type":"ContainerDied","Data":"1b75788d31c2b413c21be7642abf31e6da25b3895f319338204fd95446c71677"} Sep 05 06:32:52.968908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.968897 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2dkq4" Sep 05 06:32:52.969115 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:52.968921 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b75788d31c2b413c21be7642abf31e6da25b3895f319338204fd95446c71677" Sep 05 06:32:59.097563 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.097524 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq"] Sep 05 06:32:59.098055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.097921 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82bf35cd-11ab-4cd7-a90c-980d2630ca59" containerName="extract" Sep 05 06:32:59.098055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.097934 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="82bf35cd-11ab-4cd7-a90c-980d2630ca59" containerName="extract" Sep 05 06:32:59.098055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.097942 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82bf35cd-11ab-4cd7-a90c-980d2630ca59" containerName="pull" Sep 05 06:32:59.098055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.097948 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="82bf35cd-11ab-4cd7-a90c-980d2630ca59" containerName="pull" Sep 05 06:32:59.098055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.097961 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82bf35cd-11ab-4cd7-a90c-980d2630ca59" containerName="util" Sep 05 06:32:59.098055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.097971 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="82bf35cd-11ab-4cd7-a90c-980d2630ca59" containerName="util" Sep 05 06:32:59.098055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.098042 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="82bf35cd-11ab-4cd7-a90c-980d2630ca59" containerName="extract" Sep 05 06:32:59.101275 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.101258 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.104495 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.104462 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 05 06:32:59.104640 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.104563 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 05 06:32:59.104640 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.104565 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-plzhh\"" Sep 05 06:32:59.116925 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.116899 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq"] Sep 05 06:32:59.164876 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.164841 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrt5l\" (UniqueName: \"kubernetes.io/projected/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-kube-api-access-vrt5l\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.165022 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.164883 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.165064 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.165014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.265959 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.265923 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrt5l\" (UniqueName: \"kubernetes.io/projected/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-kube-api-access-vrt5l\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.266146 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.265984 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.266146 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.266090 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.266570 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.266544 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.266694 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.266570 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.276018 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.275997 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrt5l\" (UniqueName: \"kubernetes.io/projected/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-kube-api-access-vrt5l\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.417323 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.417213 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:32:59.693236 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:32:59.693084 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq"] Sep 05 06:32:59.695562 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:32:59.695521 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda20f85c9_3c8b_4733_8043_2bd9ccc2a70f.slice/crio-c514a6d7cfa8dbb738dc47adcb87b6e06a4df77cb5b83a49db0a4fbd85a7caa5 WatchSource:0}: Error finding container c514a6d7cfa8dbb738dc47adcb87b6e06a4df77cb5b83a49db0a4fbd85a7caa5: Status 404 returned error can't find the container with id c514a6d7cfa8dbb738dc47adcb87b6e06a4df77cb5b83a49db0a4fbd85a7caa5 Sep 05 06:33:00.004225 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.004188 2583 generic.go:356] "Generic (PLEG): container finished" podID="a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" containerID="ad4e6025bc7ebd61c8af77273aa355ce18f44ebfe177fafd70cc5ad2a82254d9" exitCode=0 Sep 05 06:33:00.004428 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.004282 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" event={"ID":"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f","Type":"ContainerDied","Data":"ad4e6025bc7ebd61c8af77273aa355ce18f44ebfe177fafd70cc5ad2a82254d9"} Sep 05 06:33:00.004428 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.004337 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" event={"ID":"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f","Type":"ContainerStarted","Data":"c514a6d7cfa8dbb738dc47adcb87b6e06a4df77cb5b83a49db0a4fbd85a7caa5"} Sep 05 06:33:00.850388 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.850303 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd"] Sep 05 06:33:00.853833 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.853817 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:00.856939 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.856917 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 05 06:33:00.857334 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.857313 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xvm2f\"" Sep 05 06:33:00.858451 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.858431 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 05 06:33:00.858569 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.858511 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 05 06:33:00.865880 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.865862 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 05 06:33:00.875979 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.875955 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd"] Sep 05 06:33:00.979732 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.979689 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10616a9b-2025-47d2-8f21-9334f0aefbdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:00.979934 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.979796 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10616a9b-2025-47d2-8f21-9334f0aefbdc-manifests\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:00.979934 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.979848 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10616a9b-2025-47d2-8f21-9334f0aefbdc-tmp\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:00.979934 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.979882 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt75j\" (UniqueName: \"kubernetes.io/projected/10616a9b-2025-47d2-8f21-9334f0aefbdc-kube-api-access-qt75j\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:00.979934 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:00.979909 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10616a9b-2025-47d2-8f21-9334f0aefbdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.020873 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.020414 2583 generic.go:356] "Generic (PLEG): container finished" podID="a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" containerID="88bc72b6dec2f39602a11abc2630bb1f81c43c236621db0e323dd5a76a41f68b" exitCode=0 Sep 05 06:33:01.020873 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.020562 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" event={"ID":"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f","Type":"ContainerDied","Data":"88bc72b6dec2f39602a11abc2630bb1f81c43c236621db0e323dd5a76a41f68b"} Sep 05 06:33:01.080632 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.080600 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10616a9b-2025-47d2-8f21-9334f0aefbdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.080745 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.080719 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10616a9b-2025-47d2-8f21-9334f0aefbdc-manifests\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.080821 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.080753 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10616a9b-2025-47d2-8f21-9334f0aefbdc-tmp\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.080821 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.080786 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt75j\" (UniqueName: \"kubernetes.io/projected/10616a9b-2025-47d2-8f21-9334f0aefbdc-kube-api-access-qt75j\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.080935 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.080821 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10616a9b-2025-47d2-8f21-9334f0aefbdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.081187 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.081160 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10616a9b-2025-47d2-8f21-9334f0aefbdc-manifests\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.083285 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.083256 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10616a9b-2025-47d2-8f21-9334f0aefbdc-tmp\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.083956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.083932 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10616a9b-2025-47d2-8f21-9334f0aefbdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.084058 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.083956 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10616a9b-2025-47d2-8f21-9334f0aefbdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.093285 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.093177 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt75j\" (UniqueName: \"kubernetes.io/projected/10616a9b-2025-47d2-8f21-9334f0aefbdc-kube-api-access-qt75j\") pod \"opendatahub-operator-controller-manager-665f9d595b-9wgfd\" (UID: \"10616a9b-2025-47d2-8f21-9334f0aefbdc\") " pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.164817 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.164725 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:01.445705 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:01.445679 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd"] Sep 05 06:33:01.447953 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:33:01.447928 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10616a9b_2025_47d2_8f21_9334f0aefbdc.slice/crio-b15ef8b4241d147b89cb961d28042368a56d5a566bf34f137f9ea9766930eccc WatchSource:0}: Error finding container b15ef8b4241d147b89cb961d28042368a56d5a566bf34f137f9ea9766930eccc: Status 404 returned error can't find the container with id b15ef8b4241d147b89cb961d28042368a56d5a566bf34f137f9ea9766930eccc Sep 05 06:33:02.028911 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:02.028863 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" event={"ID":"10616a9b-2025-47d2-8f21-9334f0aefbdc","Type":"ContainerStarted","Data":"b15ef8b4241d147b89cb961d28042368a56d5a566bf34f137f9ea9766930eccc"} Sep 05 06:33:02.031899 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:02.031871 2583 generic.go:356] "Generic (PLEG): container finished" podID="a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" containerID="36bf21696743d028109b2e4efa4ff4a95e5f565cd0c11c8dde2357e66191f945" exitCode=0 Sep 05 06:33:02.032025 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:02.031977 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" event={"ID":"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f","Type":"ContainerDied","Data":"36bf21696743d028109b2e4efa4ff4a95e5f565cd0c11c8dde2357e66191f945"} Sep 05 06:33:04.326708 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:04.326682 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:33:04.413189 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:04.413166 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrt5l\" (UniqueName: \"kubernetes.io/projected/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-kube-api-access-vrt5l\") pod \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " Sep 05 06:33:04.413285 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:04.413258 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-util\") pod \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " Sep 05 06:33:04.413285 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:04.413279 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-bundle\") pod \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\" (UID: \"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f\") " Sep 05 06:33:04.414030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:04.414007 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-bundle" (OuterVolumeSpecName: "bundle") pod "a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" (UID: "a20f85c9-3c8b-4733-8043-2bd9ccc2a70f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:33:04.415078 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:04.415052 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-kube-api-access-vrt5l" (OuterVolumeSpecName: "kube-api-access-vrt5l") pod "a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" (UID: "a20f85c9-3c8b-4733-8043-2bd9ccc2a70f"). InnerVolumeSpecName "kube-api-access-vrt5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:33:04.419812 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:04.419786 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-util" (OuterVolumeSpecName: "util") pod "a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" (UID: "a20f85c9-3c8b-4733-8043-2bd9ccc2a70f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:33:04.514751 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:04.514712 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrt5l\" (UniqueName: \"kubernetes.io/projected/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-kube-api-access-vrt5l\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:33:04.514751 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:04.514744 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-util\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:33:04.514751 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:04.514753 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a20f85c9-3c8b-4733-8043-2bd9ccc2a70f-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:33:05.051144 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:05.051110 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" event={"ID":"10616a9b-2025-47d2-8f21-9334f0aefbdc","Type":"ContainerStarted","Data":"1db20429ad3a1a28c50086c878a8982dc734cd0e16972dccdbeca72a7b4c12c1"} Sep 05 06:33:05.053135 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:05.053111 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" event={"ID":"a20f85c9-3c8b-4733-8043-2bd9ccc2a70f","Type":"ContainerDied","Data":"c514a6d7cfa8dbb738dc47adcb87b6e06a4df77cb5b83a49db0a4fbd85a7caa5"} Sep 05 06:33:05.053249 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:05.053150 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c514a6d7cfa8dbb738dc47adcb87b6e06a4df77cb5b83a49db0a4fbd85a7caa5" Sep 05 06:33:05.053249 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:05.053106 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0hg4wq" Sep 05 06:33:06.059515 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:06.059467 2583 generic.go:356] "Generic (PLEG): container finished" podID="10616a9b-2025-47d2-8f21-9334f0aefbdc" containerID="1db20429ad3a1a28c50086c878a8982dc734cd0e16972dccdbeca72a7b4c12c1" exitCode=0 Sep 05 06:33:06.059515 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:06.059519 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" event={"ID":"10616a9b-2025-47d2-8f21-9334f0aefbdc","Type":"ContainerDied","Data":"1db20429ad3a1a28c50086c878a8982dc734cd0e16972dccdbeca72a7b4c12c1"} Sep 05 06:33:07.066806 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:07.066769 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" event={"ID":"10616a9b-2025-47d2-8f21-9334f0aefbdc","Type":"ContainerStarted","Data":"eb756b890a94a880279933c4fe3b2b25dbefbfb4ea1d7ce2b59d3f011aede80d"} Sep 05 06:33:07.067227 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:07.066903 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:07.097980 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:07.097929 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" podStartSLOduration=2.79341498 podStartE2EDuration="7.097913044s" totalImagesPullingTime="3.220513402s" totalInitContainerRuntime="1.083984662s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:07.095990182 +0000 UTC m=+483.779495090" watchObservedRunningTime="2026-09-05 06:33:07.097913044 +0000 UTC m=+483.781417951" Sep 05 06:33:13.358870 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.358835 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65c4f6986c-k7gfq"] Sep 05 06:33:13.359652 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.359625 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" containerName="util" Sep 05 06:33:13.359652 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.359650 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" containerName="util" Sep 05 06:33:13.359833 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.359666 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" containerName="pull" Sep 05 06:33:13.359833 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.359675 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" containerName="pull" Sep 05 06:33:13.359833 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.359691 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" containerName="extract" Sep 05 06:33:13.359833 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.359700 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" containerName="extract" Sep 05 06:33:13.359833 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.359810 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a20f85c9-3c8b-4733-8043-2bd9ccc2a70f" containerName="extract" Sep 05 06:33:13.368062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.368037 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.373289 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.373260 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65c4f6986c-k7gfq"] Sep 05 06:33:13.492798 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.492761 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f983edf-1d0b-493d-8368-422ac363e9ea-console-oauth-config\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.492953 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.492807 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzn55\" (UniqueName: \"kubernetes.io/projected/4f983edf-1d0b-493d-8368-422ac363e9ea-kube-api-access-mzn55\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.492953 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.492829 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f983edf-1d0b-493d-8368-422ac363e9ea-console-serving-cert\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.492953 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.492848 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-service-ca\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.492953 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.492874 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-trusted-ca-bundle\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.492953 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.492910 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-oauth-serving-cert\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.492953 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.492942 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-console-config\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.594091 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.594058 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f983edf-1d0b-493d-8368-422ac363e9ea-console-serving-cert\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.594091 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.594093 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-service-ca\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.594344 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.594120 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-trusted-ca-bundle\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.594344 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.594179 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-oauth-serving-cert\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.594344 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.594223 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-console-config\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.594518 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.594371 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f983edf-1d0b-493d-8368-422ac363e9ea-console-oauth-config\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.594518 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.594445 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzn55\" (UniqueName: \"kubernetes.io/projected/4f983edf-1d0b-493d-8368-422ac363e9ea-kube-api-access-mzn55\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.594996 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.594971 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-service-ca\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.595101 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.595035 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-console-config\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.595101 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.595043 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-oauth-serving-cert\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.595193 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.595111 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f983edf-1d0b-493d-8368-422ac363e9ea-trusted-ca-bundle\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.596691 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.596665 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f983edf-1d0b-493d-8368-422ac363e9ea-console-serving-cert\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.596775 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.596689 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f983edf-1d0b-493d-8368-422ac363e9ea-console-oauth-config\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.603907 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.603885 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzn55\" (UniqueName: \"kubernetes.io/projected/4f983edf-1d0b-493d-8368-422ac363e9ea-kube-api-access-mzn55\") pod \"console-65c4f6986c-k7gfq\" (UID: \"4f983edf-1d0b-493d-8368-422ac363e9ea\") " pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.680246 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.680152 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:13.958964 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:13.958935 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65c4f6986c-k7gfq"] Sep 05 06:33:13.960432 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:33:13.960401 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f983edf_1d0b_493d_8368_422ac363e9ea.slice/crio-4fd712727f272449aaf0583a4d4d9800bd33dd729b9a8a46c146ca514aca365c WatchSource:0}: Error finding container 4fd712727f272449aaf0583a4d4d9800bd33dd729b9a8a46c146ca514aca365c: Status 404 returned error can't find the container with id 4fd712727f272449aaf0583a4d4d9800bd33dd729b9a8a46c146ca514aca365c Sep 05 06:33:14.103153 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:14.103115 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65c4f6986c-k7gfq" event={"ID":"4f983edf-1d0b-493d-8368-422ac363e9ea","Type":"ContainerStarted","Data":"741dffbc40ae10b7da70e9f35314fcf5a8ab5b9e3828bbbbd0c5fd264c93292a"} Sep 05 06:33:14.103339 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:14.103163 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65c4f6986c-k7gfq" event={"ID":"4f983edf-1d0b-493d-8368-422ac363e9ea","Type":"ContainerStarted","Data":"4fd712727f272449aaf0583a4d4d9800bd33dd729b9a8a46c146ca514aca365c"} Sep 05 06:33:14.122890 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:14.122838 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65c4f6986c-k7gfq" podStartSLOduration=1.122823907 podStartE2EDuration="1.122823907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:14.121033827 +0000 UTC m=+490.804538733" watchObservedRunningTime="2026-09-05 06:33:14.122823907 +0000 UTC m=+490.806328813" Sep 05 06:33:18.074600 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:18.074564 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-665f9d595b-9wgfd" Sep 05 06:33:23.680282 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:23.680250 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:23.680699 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:23.680519 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:23.684924 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:23.684901 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:24.162581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:24.162555 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65c4f6986c-k7gfq" Sep 05 06:33:24.245579 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:24.245544 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f87dfd46b-fddtz"] Sep 05 06:33:31.297881 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.297845 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dl59m"] Sep 05 06:33:31.301443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.301418 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:31.303614 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.303595 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 05 06:33:31.303705 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.303683 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dgnc9\"" Sep 05 06:33:31.313097 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.313070 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dl59m"] Sep 05 06:33:31.354466 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.354429 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5058292-d2de-4800-90f9-f87a10f411f6-cert\") pod \"odh-model-controller-7998c67dc4-dl59m\" (UID: \"b5058292-d2de-4800-90f9-f87a10f411f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:31.354652 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.354569 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pvjb\" (UniqueName: \"kubernetes.io/projected/b5058292-d2de-4800-90f9-f87a10f411f6-kube-api-access-9pvjb\") pod \"odh-model-controller-7998c67dc4-dl59m\" (UID: \"b5058292-d2de-4800-90f9-f87a10f411f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:31.455939 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.455908 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pvjb\" (UniqueName: \"kubernetes.io/projected/b5058292-d2de-4800-90f9-f87a10f411f6-kube-api-access-9pvjb\") pod \"odh-model-controller-7998c67dc4-dl59m\" (UID: \"b5058292-d2de-4800-90f9-f87a10f411f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:31.456119 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.455976 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5058292-d2de-4800-90f9-f87a10f411f6-cert\") pod \"odh-model-controller-7998c67dc4-dl59m\" (UID: \"b5058292-d2de-4800-90f9-f87a10f411f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:31.456119 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:33:31.456099 2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 05 06:33:31.456240 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:33:31.456164 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5058292-d2de-4800-90f9-f87a10f411f6-cert podName:b5058292-d2de-4800-90f9-f87a10f411f6 nodeName:}" failed. No retries permitted until 2026-09-05 06:33:31.956144147 +0000 UTC m=+508.639649044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b5058292-d2de-4800-90f9-f87a10f411f6-cert") pod "odh-model-controller-7998c67dc4-dl59m" (UID: "b5058292-d2de-4800-90f9-f87a10f411f6") : secret "odh-model-controller-webhook-cert" not found Sep 05 06:33:31.464466 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.464437 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pvjb\" (UniqueName: \"kubernetes.io/projected/b5058292-d2de-4800-90f9-f87a10f411f6-kube-api-access-9pvjb\") pod \"odh-model-controller-7998c67dc4-dl59m\" (UID: \"b5058292-d2de-4800-90f9-f87a10f411f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:31.960306 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.960272 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5058292-d2de-4800-90f9-f87a10f411f6-cert\") pod \"odh-model-controller-7998c67dc4-dl59m\" (UID: \"b5058292-d2de-4800-90f9-f87a10f411f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:31.962658 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:31.962631 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5058292-d2de-4800-90f9-f87a10f411f6-cert\") pod \"odh-model-controller-7998c67dc4-dl59m\" (UID: \"b5058292-d2de-4800-90f9-f87a10f411f6\") " pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:32.213159 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:32.213074 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:32.484513 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:32.484472 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dl59m"] Sep 05 06:33:32.486683 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:33:32.486655 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5058292_d2de_4800_90f9_f87a10f411f6.slice/crio-d1a2833871bc743bcfdf394c3cb5bd6764830f8054ee5e710371fce793e71202 WatchSource:0}: Error finding container d1a2833871bc743bcfdf394c3cb5bd6764830f8054ee5e710371fce793e71202: Status 404 returned error can't find the container with id d1a2833871bc743bcfdf394c3cb5bd6764830f8054ee5e710371fce793e71202 Sep 05 06:33:33.208914 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:33.208880 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" event={"ID":"b5058292-d2de-4800-90f9-f87a10f411f6","Type":"ContainerStarted","Data":"d1a2833871bc743bcfdf394c3cb5bd6764830f8054ee5e710371fce793e71202"} Sep 05 06:33:35.222076 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:35.222041 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" event={"ID":"b5058292-d2de-4800-90f9-f87a10f411f6","Type":"ContainerStarted","Data":"d7789a064cc46a2fb11abd72b249571aa2091de7e0280c1bdad91fdee019ea9e"} Sep 05 06:33:35.222423 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:35.222164 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:35.240693 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:35.240645 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" podStartSLOduration=1.5955000529999999 podStartE2EDuration="4.240630138s" totalImagesPullingTime="2.645130085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:35.239431234 +0000 UTC m=+511.922936142" watchObservedRunningTime="2026-09-05 06:33:35.240630138 +0000 UTC m=+511.924135045" Sep 05 06:33:37.848438 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:37.848404 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk"] Sep 05 06:33:37.925261 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:37.925221 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk"] Sep 05 06:33:37.925443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:37.925370 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:37.930249 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:37.930226 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 05 06:33:37.930249 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:37.930242 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 05 06:33:37.931090 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:37.931056 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 05 06:33:37.932588 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:37.932570 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wl8c2\"" Sep 05 06:33:37.932776 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:37.932618 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 05 06:33:37.933263 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:37.933246 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 05 06:33:37.933351 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:37.933271 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 05 06:33:38.014851 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.014815 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.015013 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.014862 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/370e8eb8-8963-4705-b5d6-62f22eb84326-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.015013 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.014888 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.015013 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.014907 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.015013 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.014987 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf74c\" (UniqueName: \"kubernetes.io/projected/370e8eb8-8963-4705-b5d6-62f22eb84326-kube-api-access-zf74c\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.015148 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.015023 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.015148 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.015055 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/370e8eb8-8963-4705-b5d6-62f22eb84326-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.116565 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.116456 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/370e8eb8-8963-4705-b5d6-62f22eb84326-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.116717 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.116572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.116717 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.116609 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/370e8eb8-8963-4705-b5d6-62f22eb84326-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.116717 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.116642 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.116717 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.116673 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.116717 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.116715 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf74c\" (UniqueName: \"kubernetes.io/projected/370e8eb8-8963-4705-b5d6-62f22eb84326-kube-api-access-zf74c\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.116998 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.116750 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.117528 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.117504 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.118994 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.118974 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/370e8eb8-8963-4705-b5d6-62f22eb84326-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.119411 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.119374 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/370e8eb8-8963-4705-b5d6-62f22eb84326-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.119526 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.119443 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.119526 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.119505 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.129406 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.129377 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370e8eb8-8963-4705-b5d6-62f22eb84326-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.130232 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.130192 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf74c\" (UniqueName: \"kubernetes.io/projected/370e8eb8-8963-4705-b5d6-62f22eb84326-kube-api-access-zf74c\") pod \"istiod-openshift-gateway-5677ccb49b-g8pgk\" (UID: \"370e8eb8-8963-4705-b5d6-62f22eb84326\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.235276 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.235244 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:38.481205 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.481171 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4"] Sep 05 06:33:38.484172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.484153 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.488876 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.488784 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 05 06:33:38.488876 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.488805 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 05 06:33:38.488876 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.488839 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 05 06:33:38.489358 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.489342 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 05 06:33:38.489805 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.489788 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 05 06:33:38.490073 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.490057 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kdznw\"" Sep 05 06:33:38.507891 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.507865 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4"] Sep 05 06:33:38.520203 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.520180 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-metrics-cert\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.520329 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.520249 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djb8d\" (UniqueName: \"kubernetes.io/projected/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-kube-api-access-djb8d\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.520329 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.520297 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-manager-config\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.520470 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.520403 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-cert\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.532654 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:33:38.532615 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod370e8eb8_8963_4705_b5d6_62f22eb84326.slice/crio-9f55b9337d4927be9f30a7c0617f2da699e8f9ffb33ce9d6e47ab5e0ee1d42d1 WatchSource:0}: Error finding container 9f55b9337d4927be9f30a7c0617f2da699e8f9ffb33ce9d6e47ab5e0ee1d42d1: Status 404 returned error can't find the container with id 9f55b9337d4927be9f30a7c0617f2da699e8f9ffb33ce9d6e47ab5e0ee1d42d1 Sep 05 06:33:38.535464 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.535439 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk"] Sep 05 06:33:38.621059 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.621030 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-cert\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.621197 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.621077 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-metrics-cert\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.621197 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.621116 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djb8d\" (UniqueName: \"kubernetes.io/projected/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-kube-api-access-djb8d\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.621197 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.621148 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-manager-config\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.621882 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.621861 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-manager-config\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.623540 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.623509 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-metrics-cert\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.623651 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.623579 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-cert\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.635707 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.635682 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djb8d\" (UniqueName: \"kubernetes.io/projected/56b0a15b-232a-4e7f-8c4f-ac45619c11e0-kube-api-access-djb8d\") pod \"lws-controller-manager-688d76797b-7m4d4\" (UID: \"56b0a15b-232a-4e7f-8c4f-ac45619c11e0\") " pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:38.795096 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:38.795013 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:39.084935 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:39.084908 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4"] Sep 05 06:33:39.087516 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:33:39.087471 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b0a15b_232a_4e7f_8c4f_ac45619c11e0.slice/crio-d4e2abefe5bb618214020aeef221799980012afc38f191176127bdf121f5a8ac WatchSource:0}: Error finding container d4e2abefe5bb618214020aeef221799980012afc38f191176127bdf121f5a8ac: Status 404 returned error can't find the container with id d4e2abefe5bb618214020aeef221799980012afc38f191176127bdf121f5a8ac Sep 05 06:33:39.247429 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:39.247375 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" event={"ID":"56b0a15b-232a-4e7f-8c4f-ac45619c11e0","Type":"ContainerStarted","Data":"d4e2abefe5bb618214020aeef221799980012afc38f191176127bdf121f5a8ac"} Sep 05 06:33:39.249262 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:39.249224 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" event={"ID":"370e8eb8-8963-4705-b5d6-62f22eb84326","Type":"ContainerStarted","Data":"9f55b9337d4927be9f30a7c0617f2da699e8f9ffb33ce9d6e47ab5e0ee1d42d1"} Sep 05 06:33:41.584562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:41.584230 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:33:41.584562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:41.584308 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:33:42.271428 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:42.271395 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" event={"ID":"56b0a15b-232a-4e7f-8c4f-ac45619c11e0","Type":"ContainerStarted","Data":"6c1a3628bf9e8523395428a8540e15f4635b65a24e408e6384a5e6413271b4d6"} Sep 05 06:33:42.271627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:42.271455 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:33:42.273095 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:42.273073 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" event={"ID":"370e8eb8-8963-4705-b5d6-62f22eb84326","Type":"ContainerStarted","Data":"afa78d91d45f6bafe9650d993dd07b99fca869ec9eb0ef45cab0e39bd840b7b3"} Sep 05 06:33:42.273250 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:42.273239 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:42.274979 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:42.274955 2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-g8pgk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 05 06:33:42.275087 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:42.275003 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" podUID="370e8eb8-8963-4705-b5d6-62f22eb84326" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 05 06:33:42.296804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:42.296763 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" podStartSLOduration=1.800434981 podStartE2EDuration="4.296749014s" totalImagesPullingTime="2.496314033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:42.29621232 +0000 UTC m=+518.979717228" watchObservedRunningTime="2026-09-05 06:33:42.296749014 +0000 UTC m=+518.980253983" Sep 05 06:33:42.325830 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:42.325769 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" podStartSLOduration=2.2770196289999998 podStartE2EDuration="5.325747876s" totalImagesPullingTime="3.048728247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:42.324243159 +0000 UTC m=+519.007748076" watchObservedRunningTime="2026-09-05 06:33:42.325747876 +0000 UTC m=+519.009252784" Sep 05 06:33:43.280553 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:43.280525 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g8pgk" Sep 05 06:33:46.228942 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:46.228911 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-dl59m" Sep 05 06:33:49.271215 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.271147 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f87dfd46b-fddtz" podUID="d534ebba-5b3a-48c8-976f-9f0942768cbd" containerName="console" containerID="cri-o://7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615" gracePeriod=15 Sep 05 06:33:49.515819 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.515794 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:33:49.643172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643077 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-trusted-ca-bundle\") pod \"d534ebba-5b3a-48c8-976f-9f0942768cbd\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " Sep 05 06:33:49.643172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643114 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-serving-cert\") pod \"d534ebba-5b3a-48c8-976f-9f0942768cbd\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " Sep 05 06:33:49.643172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643147 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-config\") pod \"d534ebba-5b3a-48c8-976f-9f0942768cbd\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " Sep 05 06:33:49.643172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643173 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-oauth-serving-cert\") pod \"d534ebba-5b3a-48c8-976f-9f0942768cbd\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " Sep 05 06:33:49.643506 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643245 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-oauth-config\") pod \"d534ebba-5b3a-48c8-976f-9f0942768cbd\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " Sep 05 06:33:49.643506 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643276 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-service-ca\") pod \"d534ebba-5b3a-48c8-976f-9f0942768cbd\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " Sep 05 06:33:49.643506 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643333 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5s4w2\" (UniqueName: \"kubernetes.io/projected/d534ebba-5b3a-48c8-976f-9f0942768cbd-kube-api-access-5s4w2\") pod \"d534ebba-5b3a-48c8-976f-9f0942768cbd\" (UID: \"d534ebba-5b3a-48c8-976f-9f0942768cbd\") " Sep 05 06:33:49.643670 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643645 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-config" (OuterVolumeSpecName: "console-config") pod "d534ebba-5b3a-48c8-976f-9f0942768cbd" (UID: "d534ebba-5b3a-48c8-976f-9f0942768cbd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:33:49.643724 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643652 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d534ebba-5b3a-48c8-976f-9f0942768cbd" (UID: "d534ebba-5b3a-48c8-976f-9f0942768cbd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:33:49.644023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643759 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-service-ca" (OuterVolumeSpecName: "service-ca") pod "d534ebba-5b3a-48c8-976f-9f0942768cbd" (UID: "d534ebba-5b3a-48c8-976f-9f0942768cbd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:33:49.644023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643770 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d534ebba-5b3a-48c8-976f-9f0942768cbd" (UID: "d534ebba-5b3a-48c8-976f-9f0942768cbd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:33:49.644023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643971 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-trusted-ca-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:33:49.644023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.643998 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:33:49.644023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.644013 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-oauth-serving-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:33:49.644300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.644027 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d534ebba-5b3a-48c8-976f-9f0942768cbd-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:33:49.645782 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.645755 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d534ebba-5b3a-48c8-976f-9f0942768cbd" (UID: "d534ebba-5b3a-48c8-976f-9f0942768cbd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:33:49.645782 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.645764 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d534ebba-5b3a-48c8-976f-9f0942768cbd-kube-api-access-5s4w2" (OuterVolumeSpecName: "kube-api-access-5s4w2") pod "d534ebba-5b3a-48c8-976f-9f0942768cbd" (UID: "d534ebba-5b3a-48c8-976f-9f0942768cbd"). InnerVolumeSpecName "kube-api-access-5s4w2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:33:49.645972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.645840 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d534ebba-5b3a-48c8-976f-9f0942768cbd" (UID: "d534ebba-5b3a-48c8-976f-9f0942768cbd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:33:49.744814 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.744773 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-oauth-config\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:33:49.744814 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.744806 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5s4w2\" (UniqueName: \"kubernetes.io/projected/d534ebba-5b3a-48c8-976f-9f0942768cbd-kube-api-access-5s4w2\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:33:49.744814 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:49.744816 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d534ebba-5b3a-48c8-976f-9f0942768cbd-console-serving-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:33:50.317752 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:50.317715 2583 generic.go:356] "Generic (PLEG): container finished" podID="d534ebba-5b3a-48c8-976f-9f0942768cbd" containerID="7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615" exitCode=0 Sep 05 06:33:50.318191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:50.317761 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f87dfd46b-fddtz" Sep 05 06:33:50.318191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:50.317781 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f87dfd46b-fddtz" event={"ID":"d534ebba-5b3a-48c8-976f-9f0942768cbd","Type":"ContainerDied","Data":"7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615"} Sep 05 06:33:50.318191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:50.317820 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f87dfd46b-fddtz" event={"ID":"d534ebba-5b3a-48c8-976f-9f0942768cbd","Type":"ContainerDied","Data":"2ddf50e8e5cfdc9d9960e4d8ac2606cc20059da959ee2219d2850692f8bed845"} Sep 05 06:33:50.318191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:50.317839 2583 scope.go:117] "RemoveContainer" containerID="7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615" Sep 05 06:33:50.326738 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:50.326720 2583 scope.go:117] "RemoveContainer" containerID="7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615" Sep 05 06:33:50.327009 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:33:50.326990 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615\": container with ID starting with 7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615 not found: ID does not exist" containerID="7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615" Sep 05 06:33:50.327066 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:50.327017 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615"} err="failed to get container status \"7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615\": rpc error: code = NotFound desc = could not find container \"7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615\": container with ID starting with 7622efefc422eba5947b34638a5a1089adeb83d44ef43177167e7ab62d094615 not found: ID does not exist" Sep 05 06:33:50.336251 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:50.336221 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f87dfd46b-fddtz"] Sep 05 06:33:50.342595 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:50.342575 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f87dfd46b-fddtz"] Sep 05 06:33:51.823086 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:51.823044 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d534ebba-5b3a-48c8-976f-9f0942768cbd" path="/var/lib/kubelet/pods/d534ebba-5b3a-48c8-976f-9f0942768cbd/volumes" Sep 05 06:33:53.281633 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:33:53.281602 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-688d76797b-7m4d4" Sep 05 06:34:36.334961 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.334926 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf"] Sep 05 06:34:36.335412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.335362 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d534ebba-5b3a-48c8-976f-9f0942768cbd" containerName="console" Sep 05 06:34:36.335412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.335376 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d534ebba-5b3a-48c8-976f-9f0942768cbd" containerName="console" Sep 05 06:34:36.335515 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.335491 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d534ebba-5b3a-48c8-976f-9f0942768cbd" containerName="console" Sep 05 06:34:36.338845 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.338830 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.341270 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.341248 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 05 06:34:36.341392 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.341272 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-plzhh\"" Sep 05 06:34:36.342179 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.342165 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 05 06:34:36.352053 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.352029 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf"] Sep 05 06:34:36.427575 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.427537 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72whs\" (UniqueName: \"kubernetes.io/projected/abd25890-9f1b-4224-a82d-0d7e4be73a3b-kube-api-access-72whs\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.427738 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.427600 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.427738 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.427657 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.431542 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.431518 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z"] Sep 05 06:34:36.435493 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.435463 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.444465 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.444444 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z"] Sep 05 06:34:36.528911 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.528884 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whtq9\" (UniqueName: \"kubernetes.io/projected/29a183e6-de88-4374-ba51-44e225314335-kube-api-access-whtq9\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.529040 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.528919 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.529040 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.528948 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.529165 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.529044 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72whs\" (UniqueName: \"kubernetes.io/projected/abd25890-9f1b-4224-a82d-0d7e4be73a3b-kube-api-access-72whs\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.529165 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.529086 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.529165 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.529134 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.529345 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.529323 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.529505 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.529465 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.531937 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.531917 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r"] Sep 05 06:34:36.535753 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.535738 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.538025 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.537996 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72whs\" (UniqueName: \"kubernetes.io/projected/abd25890-9f1b-4224-a82d-0d7e4be73a3b-kube-api-access-72whs\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.546348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.546328 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r"] Sep 05 06:34:36.630430 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.630362 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.630430 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.630396 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtqz6\" (UniqueName: \"kubernetes.io/projected/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-kube-api-access-gtqz6\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.630624 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.630431 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.630624 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.630517 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.630624 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.630566 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whtq9\" (UniqueName: \"kubernetes.io/projected/29a183e6-de88-4374-ba51-44e225314335-kube-api-access-whtq9\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.630624 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.630608 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.630768 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.630750 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.630921 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.630905 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.644628 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.644602 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq"] Sep 05 06:34:36.645745 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.645725 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whtq9\" (UniqueName: \"kubernetes.io/projected/29a183e6-de88-4374-ba51-44e225314335-kube-api-access-whtq9\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.648565 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.648545 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:36.648616 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.648577 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:36.657911 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.657879 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq"] Sep 05 06:34:36.731163 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.731132 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:36.731315 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.731181 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:36.731315 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.731291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.731443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.731326 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtqz6\" (UniqueName: \"kubernetes.io/projected/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-kube-api-access-gtqz6\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.731443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.731390 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.731443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.731423 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfjq9\" (UniqueName: \"kubernetes.io/projected/9194104c-c676-426f-a7ab-f0355d59c4b8-kube-api-access-pfjq9\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:36.731709 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.731690 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.731791 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.731771 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.741359 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.741333 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtqz6\" (UniqueName: \"kubernetes.io/projected/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-kube-api-access-gtqz6\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.745093 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.745069 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:36.832895 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.832861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:36.833061 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.832962 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfjq9\" (UniqueName: \"kubernetes.io/projected/9194104c-c676-426f-a7ab-f0355d59c4b8-kube-api-access-pfjq9\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:36.833061 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.833009 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:36.833319 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.833296 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:36.833395 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.833337 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:36.842842 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.842815 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfjq9\" (UniqueName: \"kubernetes.io/projected/9194104c-c676-426f-a7ab-f0355d59c4b8-kube-api-access-pfjq9\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:36.854592 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.854576 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:36.929579 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.929552 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf"] Sep 05 06:34:36.931739 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:34:36.931706 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd25890_9f1b_4224_a82d_0d7e4be73a3b.slice/crio-cce43e8f3d97f383ade9ee9c956d7f3574914ace14af7da0d888082572f79e52 WatchSource:0}: Error finding container cce43e8f3d97f383ade9ee9c956d7f3574914ace14af7da0d888082572f79e52: Status 404 returned error can't find the container with id cce43e8f3d97f383ade9ee9c956d7f3574914ace14af7da0d888082572f79e52 Sep 05 06:34:36.979116 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:36.979096 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:37.025917 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.025887 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z"] Sep 05 06:34:37.026406 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:34:37.026378 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a183e6_de88_4374_ba51_44e225314335.slice/crio-ea6f31dc48441ea2408fd3144499c2bded2515ec96544dfc454007cad7735865 WatchSource:0}: Error finding container ea6f31dc48441ea2408fd3144499c2bded2515ec96544dfc454007cad7735865: Status 404 returned error can't find the container with id ea6f31dc48441ea2408fd3144499c2bded2515ec96544dfc454007cad7735865 Sep 05 06:34:37.137051 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.137025 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r"] Sep 05 06:34:37.139248 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:34:37.139225 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1055b4a2_bc2f_478b_bfa4_2aa18f40b16e.slice/crio-14133b5fae7fdb57b166ba5e6300abac2c3f1995312365de536d06a114114862 WatchSource:0}: Error finding container 14133b5fae7fdb57b166ba5e6300abac2c3f1995312365de536d06a114114862: Status 404 returned error can't find the container with id 14133b5fae7fdb57b166ba5e6300abac2c3f1995312365de536d06a114114862 Sep 05 06:34:37.264056 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.264025 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq"] Sep 05 06:34:37.265214 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:34:37.265188 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9194104c_c676_426f_a7ab_f0355d59c4b8.slice/crio-43c40d152115cd6e5a9953a892d63bc9ff5a52110f9535cd2b170a0f86da6e20 WatchSource:0}: Error finding container 43c40d152115cd6e5a9953a892d63bc9ff5a52110f9535cd2b170a0f86da6e20: Status 404 returned error can't find the container with id 43c40d152115cd6e5a9953a892d63bc9ff5a52110f9535cd2b170a0f86da6e20 Sep 05 06:34:37.556701 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.556664 2583 generic.go:356] "Generic (PLEG): container finished" podID="9194104c-c676-426f-a7ab-f0355d59c4b8" containerID="8bb77c8585d38261d68f3e06e34c5b8f347eb2841e7289aa051001c1969f6cc0" exitCode=0 Sep 05 06:34:37.557115 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.556738 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" event={"ID":"9194104c-c676-426f-a7ab-f0355d59c4b8","Type":"ContainerDied","Data":"8bb77c8585d38261d68f3e06e34c5b8f347eb2841e7289aa051001c1969f6cc0"} Sep 05 06:34:37.557115 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.556776 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" event={"ID":"9194104c-c676-426f-a7ab-f0355d59c4b8","Type":"ContainerStarted","Data":"43c40d152115cd6e5a9953a892d63bc9ff5a52110f9535cd2b170a0f86da6e20"} Sep 05 06:34:37.558386 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.558362 2583 generic.go:356] "Generic (PLEG): container finished" podID="1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" containerID="ab01f6ef049926eb461d25b8098bc8d22c56c472c610b7492940bc4e9d6d2b1f" exitCode=0 Sep 05 06:34:37.558504 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.558451 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" event={"ID":"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e","Type":"ContainerDied","Data":"ab01f6ef049926eb461d25b8098bc8d22c56c472c610b7492940bc4e9d6d2b1f"} Sep 05 06:34:37.558627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.558528 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" event={"ID":"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e","Type":"ContainerStarted","Data":"14133b5fae7fdb57b166ba5e6300abac2c3f1995312365de536d06a114114862"} Sep 05 06:34:37.560346 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.560322 2583 generic.go:356] "Generic (PLEG): container finished" podID="29a183e6-de88-4374-ba51-44e225314335" containerID="a789c09282dd0650b99cb16ad9fdcd70c7084b68265715e05f894a90515e3d79" exitCode=0 Sep 05 06:34:37.560433 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.560407 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" event={"ID":"29a183e6-de88-4374-ba51-44e225314335","Type":"ContainerDied","Data":"a789c09282dd0650b99cb16ad9fdcd70c7084b68265715e05f894a90515e3d79"} Sep 05 06:34:37.560508 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.560463 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" event={"ID":"29a183e6-de88-4374-ba51-44e225314335","Type":"ContainerStarted","Data":"ea6f31dc48441ea2408fd3144499c2bded2515ec96544dfc454007cad7735865"} Sep 05 06:34:37.562187 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.562164 2583 generic.go:356] "Generic (PLEG): container finished" podID="abd25890-9f1b-4224-a82d-0d7e4be73a3b" containerID="2522ffbc61c6e1d7a57f6454345b0a247f6fe0c8b33fae69ce98603d1d1f3f3d" exitCode=0 Sep 05 06:34:37.562273 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.562193 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" event={"ID":"abd25890-9f1b-4224-a82d-0d7e4be73a3b","Type":"ContainerDied","Data":"2522ffbc61c6e1d7a57f6454345b0a247f6fe0c8b33fae69ce98603d1d1f3f3d"} Sep 05 06:34:37.562273 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:37.562223 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" event={"ID":"abd25890-9f1b-4224-a82d-0d7e4be73a3b","Type":"ContainerStarted","Data":"cce43e8f3d97f383ade9ee9c956d7f3574914ace14af7da0d888082572f79e52"} Sep 05 06:34:38.570622 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:38.570590 2583 generic.go:356] "Generic (PLEG): container finished" podID="1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" containerID="bcae4981e219996676dd472f283be6686b7162c48f696eae32b57a5127c7e92c" exitCode=0 Sep 05 06:34:38.570963 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:38.570683 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" event={"ID":"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e","Type":"ContainerDied","Data":"bcae4981e219996676dd472f283be6686b7162c48f696eae32b57a5127c7e92c"} Sep 05 06:34:38.573083 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:38.573061 2583 generic.go:356] "Generic (PLEG): container finished" podID="abd25890-9f1b-4224-a82d-0d7e4be73a3b" containerID="62346b894ef84729107b9471f97b145ca032af8d25b88334cf61d49bf74ab090" exitCode=0 Sep 05 06:34:38.573184 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:38.573132 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" event={"ID":"abd25890-9f1b-4224-a82d-0d7e4be73a3b","Type":"ContainerDied","Data":"62346b894ef84729107b9471f97b145ca032af8d25b88334cf61d49bf74ab090"} Sep 05 06:34:39.580005 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:39.579976 2583 generic.go:356] "Generic (PLEG): container finished" podID="9194104c-c676-426f-a7ab-f0355d59c4b8" containerID="3363cf44d53bdf01bbd0fb2ed1790d72024f241f377d6a4607b218a4d19b2c04" exitCode=0 Sep 05 06:34:39.580341 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:39.580056 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" event={"ID":"9194104c-c676-426f-a7ab-f0355d59c4b8","Type":"ContainerDied","Data":"3363cf44d53bdf01bbd0fb2ed1790d72024f241f377d6a4607b218a4d19b2c04"} Sep 05 06:34:39.582400 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:39.582377 2583 generic.go:356] "Generic (PLEG): container finished" podID="1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" containerID="06d8e9078355aef45418f369f66908ca16d641b5f7c4c19630860060dfbc4ed6" exitCode=0 Sep 05 06:34:39.582530 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:39.582509 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" event={"ID":"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e","Type":"ContainerDied","Data":"06d8e9078355aef45418f369f66908ca16d641b5f7c4c19630860060dfbc4ed6"} Sep 05 06:34:39.584201 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:39.584180 2583 generic.go:356] "Generic (PLEG): container finished" podID="29a183e6-de88-4374-ba51-44e225314335" containerID="c8476ee8aaa4be471f6cf6f120949bfaceb07bab92b7f7c29329178770edd063" exitCode=0 Sep 05 06:34:39.584298 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:39.584262 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" event={"ID":"29a183e6-de88-4374-ba51-44e225314335","Type":"ContainerDied","Data":"c8476ee8aaa4be471f6cf6f120949bfaceb07bab92b7f7c29329178770edd063"} Sep 05 06:34:39.586702 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:39.586679 2583 generic.go:356] "Generic (PLEG): container finished" podID="abd25890-9f1b-4224-a82d-0d7e4be73a3b" containerID="3814067abfa4f049e009baeb899de0b309bebd5d2533c639d425146f74f34e71" exitCode=0 Sep 05 06:34:39.586788 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:39.586707 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" event={"ID":"abd25890-9f1b-4224-a82d-0d7e4be73a3b","Type":"ContainerDied","Data":"3814067abfa4f049e009baeb899de0b309bebd5d2533c639d425146f74f34e71"} Sep 05 06:34:40.594500 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.594445 2583 generic.go:356] "Generic (PLEG): container finished" podID="29a183e6-de88-4374-ba51-44e225314335" containerID="fc6d7b4aafd9207fadf53a436acbdd83c5f1ec4a315bba29b28231c946aef47a" exitCode=0 Sep 05 06:34:40.594960 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.594531 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" event={"ID":"29a183e6-de88-4374-ba51-44e225314335","Type":"ContainerDied","Data":"fc6d7b4aafd9207fadf53a436acbdd83c5f1ec4a315bba29b28231c946aef47a"} Sep 05 06:34:40.596470 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.596446 2583 generic.go:356] "Generic (PLEG): container finished" podID="9194104c-c676-426f-a7ab-f0355d59c4b8" containerID="0cc70805db94179b45078de5b074502ad810d46c16d7fe3ed79d22d64a6863de" exitCode=0 Sep 05 06:34:40.596618 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.596507 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" event={"ID":"9194104c-c676-426f-a7ab-f0355d59c4b8","Type":"ContainerDied","Data":"0cc70805db94179b45078de5b074502ad810d46c16d7fe3ed79d22d64a6863de"} Sep 05 06:34:40.748877 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.748846 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:40.751590 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.751567 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:40.869385 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.869296 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-util\") pod \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " Sep 05 06:34:40.869385 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.869339 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-util\") pod \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " Sep 05 06:34:40.869385 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.869378 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtqz6\" (UniqueName: \"kubernetes.io/projected/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-kube-api-access-gtqz6\") pod \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " Sep 05 06:34:40.869662 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.869496 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-bundle\") pod \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " Sep 05 06:34:40.869662 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.869515 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-bundle\") pod \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\" (UID: \"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e\") " Sep 05 06:34:40.869662 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.869537 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72whs\" (UniqueName: \"kubernetes.io/projected/abd25890-9f1b-4224-a82d-0d7e4be73a3b-kube-api-access-72whs\") pod \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\" (UID: \"abd25890-9f1b-4224-a82d-0d7e4be73a3b\") " Sep 05 06:34:40.870068 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.870034 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-bundle" (OuterVolumeSpecName: "bundle") pod "1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" (UID: "1055b4a2-bc2f-478b-bfa4-2aa18f40b16e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:34:40.870196 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.870092 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-bundle" (OuterVolumeSpecName: "bundle") pod "abd25890-9f1b-4224-a82d-0d7e4be73a3b" (UID: "abd25890-9f1b-4224-a82d-0d7e4be73a3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:34:40.871696 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.871674 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-kube-api-access-gtqz6" (OuterVolumeSpecName: "kube-api-access-gtqz6") pod "1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" (UID: "1055b4a2-bc2f-478b-bfa4-2aa18f40b16e"). InnerVolumeSpecName "kube-api-access-gtqz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:34:40.871811 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.871768 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abd25890-9f1b-4224-a82d-0d7e4be73a3b-kube-api-access-72whs" (OuterVolumeSpecName: "kube-api-access-72whs") pod "abd25890-9f1b-4224-a82d-0d7e4be73a3b" (UID: "abd25890-9f1b-4224-a82d-0d7e4be73a3b"). InnerVolumeSpecName "kube-api-access-72whs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:34:40.874597 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.874555 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-util" (OuterVolumeSpecName: "util") pod "1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" (UID: "1055b4a2-bc2f-478b-bfa4-2aa18f40b16e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:34:40.875129 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.875104 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-util" (OuterVolumeSpecName: "util") pod "abd25890-9f1b-4224-a82d-0d7e4be73a3b" (UID: "abd25890-9f1b-4224-a82d-0d7e4be73a3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:34:40.970241 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.970207 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:40.970241 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.970238 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:40.970424 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.970250 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72whs\" (UniqueName: \"kubernetes.io/projected/abd25890-9f1b-4224-a82d-0d7e4be73a3b-kube-api-access-72whs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:40.970424 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.970261 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-util\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:40.970424 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.970269 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abd25890-9f1b-4224-a82d-0d7e4be73a3b-util\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:40.970424 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:40.970277 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtqz6\" (UniqueName: \"kubernetes.io/projected/1055b4a2-bc2f-478b-bfa4-2aa18f40b16e-kube-api-access-gtqz6\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:41.603890 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.603861 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" Sep 05 06:34:41.603890 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.603870 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cv6gtf" event={"ID":"abd25890-9f1b-4224-a82d-0d7e4be73a3b","Type":"ContainerDied","Data":"cce43e8f3d97f383ade9ee9c956d7f3574914ace14af7da0d888082572f79e52"} Sep 05 06:34:41.604489 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.603925 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cce43e8f3d97f383ade9ee9c956d7f3574914ace14af7da0d888082572f79e52" Sep 05 06:34:41.605875 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.605856 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" Sep 05 06:34:41.605983 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.605887 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbhkr8r" event={"ID":"1055b4a2-bc2f-478b-bfa4-2aa18f40b16e","Type":"ContainerDied","Data":"14133b5fae7fdb57b166ba5e6300abac2c3f1995312365de536d06a114114862"} Sep 05 06:34:41.605983 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.605923 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14133b5fae7fdb57b166ba5e6300abac2c3f1995312365de536d06a114114862" Sep 05 06:34:41.737055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.737030 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:41.764461 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.764438 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:41.877240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.877164 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whtq9\" (UniqueName: \"kubernetes.io/projected/29a183e6-de88-4374-ba51-44e225314335-kube-api-access-whtq9\") pod \"29a183e6-de88-4374-ba51-44e225314335\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " Sep 05 06:34:41.877240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.877213 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-util\") pod \"9194104c-c676-426f-a7ab-f0355d59c4b8\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " Sep 05 06:34:41.877457 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.877260 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-bundle\") pod \"29a183e6-de88-4374-ba51-44e225314335\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " Sep 05 06:34:41.877457 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.877305 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfjq9\" (UniqueName: \"kubernetes.io/projected/9194104c-c676-426f-a7ab-f0355d59c4b8-kube-api-access-pfjq9\") pod \"9194104c-c676-426f-a7ab-f0355d59c4b8\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " Sep 05 06:34:41.877457 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.877339 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-util\") pod \"29a183e6-de88-4374-ba51-44e225314335\" (UID: \"29a183e6-de88-4374-ba51-44e225314335\") " Sep 05 06:34:41.877457 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.877383 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-bundle\") pod \"9194104c-c676-426f-a7ab-f0355d59c4b8\" (UID: \"9194104c-c676-426f-a7ab-f0355d59c4b8\") " Sep 05 06:34:41.878251 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.878179 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-bundle" (OuterVolumeSpecName: "bundle") pod "29a183e6-de88-4374-ba51-44e225314335" (UID: "29a183e6-de88-4374-ba51-44e225314335"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:34:41.878251 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.878215 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-bundle" (OuterVolumeSpecName: "bundle") pod "9194104c-c676-426f-a7ab-f0355d59c4b8" (UID: "9194104c-c676-426f-a7ab-f0355d59c4b8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:34:41.878436 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.878316 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:41.878436 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.878333 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-bundle\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:41.879469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.879449 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a183e6-de88-4374-ba51-44e225314335-kube-api-access-whtq9" (OuterVolumeSpecName: "kube-api-access-whtq9") pod "29a183e6-de88-4374-ba51-44e225314335" (UID: "29a183e6-de88-4374-ba51-44e225314335"). InnerVolumeSpecName "kube-api-access-whtq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:34:41.879593 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.879574 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9194104c-c676-426f-a7ab-f0355d59c4b8-kube-api-access-pfjq9" (OuterVolumeSpecName: "kube-api-access-pfjq9") pod "9194104c-c676-426f-a7ab-f0355d59c4b8" (UID: "9194104c-c676-426f-a7ab-f0355d59c4b8"). InnerVolumeSpecName "kube-api-access-pfjq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:34:41.883519 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.883493 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-util" (OuterVolumeSpecName: "util") pod "29a183e6-de88-4374-ba51-44e225314335" (UID: "29a183e6-de88-4374-ba51-44e225314335"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:34:41.886042 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.886019 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-util" (OuterVolumeSpecName: "util") pod "9194104c-c676-426f-a7ab-f0355d59c4b8" (UID: "9194104c-c676-426f-a7ab-f0355d59c4b8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:34:41.979395 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.979370 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9194104c-c676-426f-a7ab-f0355d59c4b8-util\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:41.979395 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.979392 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfjq9\" (UniqueName: \"kubernetes.io/projected/9194104c-c676-426f-a7ab-f0355d59c4b8-kube-api-access-pfjq9\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:41.979395 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.979402 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29a183e6-de88-4374-ba51-44e225314335-util\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:41.979624 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:41.979411 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whtq9\" (UniqueName: \"kubernetes.io/projected/29a183e6-de88-4374-ba51-44e225314335-kube-api-access-whtq9\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:34:42.613092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:42.613063 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" Sep 05 06:34:42.613557 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:42.613062 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0s2rwq" event={"ID":"9194104c-c676-426f-a7ab-f0355d59c4b8","Type":"ContainerDied","Data":"43c40d152115cd6e5a9953a892d63bc9ff5a52110f9535cd2b170a0f86da6e20"} Sep 05 06:34:42.613557 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:42.613205 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43c40d152115cd6e5a9953a892d63bc9ff5a52110f9535cd2b170a0f86da6e20" Sep 05 06:34:42.615004 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:42.614981 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" event={"ID":"29a183e6-de88-4374-ba51-44e225314335","Type":"ContainerDied","Data":"ea6f31dc48441ea2408fd3144499c2bded2515ec96544dfc454007cad7735865"} Sep 05 06:34:42.615108 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:42.615018 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870537857z" Sep 05 06:34:42.615108 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:34:42.615021 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea6f31dc48441ea2408fd3144499c2bded2515ec96544dfc454007cad7735865" Sep 05 06:35:03.773390 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:35:03.773364 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:35:03.773897 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:35:03.773647 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:35:03.780415 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:35:03.780391 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:35:03.780415 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:35:03.780408 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:36:01.460455 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460423 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-w26mx"] Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460806 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a183e6-de88-4374-ba51-44e225314335" containerName="util" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460817 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a183e6-de88-4374-ba51-44e225314335" containerName="util" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460827 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abd25890-9f1b-4224-a82d-0d7e4be73a3b" containerName="extract" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460832 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="abd25890-9f1b-4224-a82d-0d7e4be73a3b" containerName="extract" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460841 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" containerName="util" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460847 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" containerName="util" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460854 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a183e6-de88-4374-ba51-44e225314335" containerName="pull" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460859 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a183e6-de88-4374-ba51-44e225314335" containerName="pull" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460872 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abd25890-9f1b-4224-a82d-0d7e4be73a3b" containerName="util" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460877 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="abd25890-9f1b-4224-a82d-0d7e4be73a3b" containerName="util" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460883 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9194104c-c676-426f-a7ab-f0355d59c4b8" containerName="pull" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460888 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9194104c-c676-426f-a7ab-f0355d59c4b8" containerName="pull" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460897 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9194104c-c676-426f-a7ab-f0355d59c4b8" containerName="util" Sep 05 06:36:01.460908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460911 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9194104c-c676-426f-a7ab-f0355d59c4b8" containerName="util" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460923 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abd25890-9f1b-4224-a82d-0d7e4be73a3b" containerName="pull" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460928 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="abd25890-9f1b-4224-a82d-0d7e4be73a3b" containerName="pull" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460934 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" containerName="pull" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460939 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" containerName="pull" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460947 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" containerName="extract" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460952 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" containerName="extract" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460959 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9194104c-c676-426f-a7ab-f0355d59c4b8" containerName="extract" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460964 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9194104c-c676-426f-a7ab-f0355d59c4b8" containerName="extract" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460969 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a183e6-de88-4374-ba51-44e225314335" containerName="extract" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.460974 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a183e6-de88-4374-ba51-44e225314335" containerName="extract" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.461026 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9194104c-c676-426f-a7ab-f0355d59c4b8" containerName="extract" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.461037 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1055b4a2-bc2f-478b-bfa4-2aa18f40b16e" containerName="extract" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.461043 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="abd25890-9f1b-4224-a82d-0d7e4be73a3b" containerName="extract" Sep 05 06:36:01.461374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.461050 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a183e6-de88-4374-ba51-44e225314335" containerName="extract" Sep 05 06:36:01.464049 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.464034 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:01.466658 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.466632 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 05 06:36:01.466658 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.466657 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-llklw\"" Sep 05 06:36:01.467643 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.467625 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 05 06:36:01.467759 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.467680 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 05 06:36:01.473880 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.473856 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-w26mx"] Sep 05 06:36:01.552845 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.552806 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8mv8\" (UniqueName: \"kubernetes.io/projected/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-kube-api-access-w8mv8\") pod \"limitador-limitador-859c4d96d6-w26mx\" (UID: \"0ff6ec99-1b4b-41d7-bc4c-a92656195af2\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:01.553037 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.552882 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-config-file\") pod \"limitador-limitador-859c4d96d6-w26mx\" (UID: \"0ff6ec99-1b4b-41d7-bc4c-a92656195af2\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:01.555452 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.555424 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-w26mx"] Sep 05 06:36:01.653744 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.653703 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mv8\" (UniqueName: \"kubernetes.io/projected/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-kube-api-access-w8mv8\") pod \"limitador-limitador-859c4d96d6-w26mx\" (UID: \"0ff6ec99-1b4b-41d7-bc4c-a92656195af2\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:01.653910 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.653751 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-config-file\") pod \"limitador-limitador-859c4d96d6-w26mx\" (UID: \"0ff6ec99-1b4b-41d7-bc4c-a92656195af2\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:01.654343 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.654323 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-config-file\") pod \"limitador-limitador-859c4d96d6-w26mx\" (UID: \"0ff6ec99-1b4b-41d7-bc4c-a92656195af2\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:01.662431 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.662408 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8mv8\" (UniqueName: \"kubernetes.io/projected/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-kube-api-access-w8mv8\") pod \"limitador-limitador-859c4d96d6-w26mx\" (UID: \"0ff6ec99-1b4b-41d7-bc4c-a92656195af2\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:01.775706 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:01.775677 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:02.053174 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:02.053134 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-w26mx"] Sep 05 06:36:02.053890 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:36:02.053869 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ff6ec99_1b4b_41d7_bc4c_a92656195af2.slice/crio-1dafca69655280b01d917d2ab38a860da3508b62155841c3e39da248f33b8c8d WatchSource:0}: Error finding container 1dafca69655280b01d917d2ab38a860da3508b62155841c3e39da248f33b8c8d: Status 404 returned error can't find the container with id 1dafca69655280b01d917d2ab38a860da3508b62155841c3e39da248f33b8c8d Sep 05 06:36:03.045688 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:03.045651 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" event={"ID":"0ff6ec99-1b4b-41d7-bc4c-a92656195af2","Type":"ContainerStarted","Data":"1dafca69655280b01d917d2ab38a860da3508b62155841c3e39da248f33b8c8d"} Sep 05 06:36:06.066849 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:06.066804 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" event={"ID":"0ff6ec99-1b4b-41d7-bc4c-a92656195af2","Type":"ContainerStarted","Data":"1748cc97afa21dcaf7d5fc75bb6c6ae1252197580267de9dc0405cc141469036"} Sep 05 06:36:06.067229 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:06.066886 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:06.085219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:06.085167 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" podStartSLOduration=1.7326244370000001 podStartE2EDuration="5.08515013s" totalImagesPullingTime="3.352525693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:36:06.08431091 +0000 UTC m=+662.767815817" watchObservedRunningTime="2026-09-05 06:36:06.08515013 +0000 UTC m=+662.768655046" Sep 05 06:36:15.577022 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:15.576989 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-w26mx"] Sep 05 06:36:15.577496 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:15.577189 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" podUID="0ff6ec99-1b4b-41d7-bc4c-a92656195af2" containerName="limitador" containerID="cri-o://1748cc97afa21dcaf7d5fc75bb6c6ae1252197580267de9dc0405cc141469036" gracePeriod=30 Sep 05 06:36:15.577814 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:15.577779 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:16.122196 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.122163 2583 generic.go:356] "Generic (PLEG): container finished" podID="0ff6ec99-1b4b-41d7-bc4c-a92656195af2" containerID="1748cc97afa21dcaf7d5fc75bb6c6ae1252197580267de9dc0405cc141469036" exitCode=0 Sep 05 06:36:16.122347 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.122238 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" event={"ID":"0ff6ec99-1b4b-41d7-bc4c-a92656195af2","Type":"ContainerDied","Data":"1748cc97afa21dcaf7d5fc75bb6c6ae1252197580267de9dc0405cc141469036"} Sep 05 06:36:16.122347 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.122302 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" event={"ID":"0ff6ec99-1b4b-41d7-bc4c-a92656195af2","Type":"ContainerDied","Data":"1dafca69655280b01d917d2ab38a860da3508b62155841c3e39da248f33b8c8d"} Sep 05 06:36:16.122347 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.122323 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1dafca69655280b01d917d2ab38a860da3508b62155841c3e39da248f33b8c8d" Sep 05 06:36:16.129420 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.129392 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:16.178550 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.178436 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-config-file\") pod \"0ff6ec99-1b4b-41d7-bc4c-a92656195af2\" (UID: \"0ff6ec99-1b4b-41d7-bc4c-a92656195af2\") " Sep 05 06:36:16.178690 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.178586 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8mv8\" (UniqueName: \"kubernetes.io/projected/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-kube-api-access-w8mv8\") pod \"0ff6ec99-1b4b-41d7-bc4c-a92656195af2\" (UID: \"0ff6ec99-1b4b-41d7-bc4c-a92656195af2\") " Sep 05 06:36:16.178817 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.178794 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-config-file" (OuterVolumeSpecName: "config-file") pod "0ff6ec99-1b4b-41d7-bc4c-a92656195af2" (UID: "0ff6ec99-1b4b-41d7-bc4c-a92656195af2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:36:16.180734 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.180704 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-kube-api-access-w8mv8" (OuterVolumeSpecName: "kube-api-access-w8mv8") pod "0ff6ec99-1b4b-41d7-bc4c-a92656195af2" (UID: "0ff6ec99-1b4b-41d7-bc4c-a92656195af2"). InnerVolumeSpecName "kube-api-access-w8mv8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:36:16.279617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.279583 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8mv8\" (UniqueName: \"kubernetes.io/projected/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-kube-api-access-w8mv8\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:36:16.279617 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:16.279611 2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0ff6ec99-1b4b-41d7-bc4c-a92656195af2-config-file\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:36:17.133918 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:17.128157 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-w26mx" Sep 05 06:36:17.155169 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:17.155142 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-w26mx"] Sep 05 06:36:17.159264 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:17.159238 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-w26mx"] Sep 05 06:36:17.822094 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:17.822053 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ff6ec99-1b4b-41d7-bc4c-a92656195af2" path="/var/lib/kubelet/pods/0ff6ec99-1b4b-41d7-bc4c-a92656195af2/volumes" Sep 05 06:36:20.180266 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.180236 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j4j25"] Sep 05 06:36:20.180672 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.180624 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ff6ec99-1b4b-41d7-bc4c-a92656195af2" containerName="limitador" Sep 05 06:36:20.180672 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.180637 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ff6ec99-1b4b-41d7-bc4c-a92656195af2" containerName="limitador" Sep 05 06:36:20.180755 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.180722 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ff6ec99-1b4b-41d7-bc4c-a92656195af2" containerName="limitador" Sep 05 06:36:20.184062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.184040 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" Sep 05 06:36:20.186755 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.186729 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 05 06:36:20.186882 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.186769 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 05 06:36:20.187800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.187777 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 05 06:36:20.187800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.187796 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-2rfc7\"" Sep 05 06:36:20.192778 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.192748 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j4j25"] Sep 05 06:36:20.315331 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.315292 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0ff1db7-eeb9-4315-a403-cf87b73f3b43-data\") pod \"postgres-69fb4594cf-j4j25\" (UID: \"e0ff1db7-eeb9-4315-a403-cf87b73f3b43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" Sep 05 06:36:20.315331 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.315327 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdpkq\" (UniqueName: \"kubernetes.io/projected/e0ff1db7-eeb9-4315-a403-cf87b73f3b43-kube-api-access-qdpkq\") pod \"postgres-69fb4594cf-j4j25\" (UID: \"e0ff1db7-eeb9-4315-a403-cf87b73f3b43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" Sep 05 06:36:20.416392 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.416350 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0ff1db7-eeb9-4315-a403-cf87b73f3b43-data\") pod \"postgres-69fb4594cf-j4j25\" (UID: \"e0ff1db7-eeb9-4315-a403-cf87b73f3b43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" Sep 05 06:36:20.416392 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.416391 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdpkq\" (UniqueName: \"kubernetes.io/projected/e0ff1db7-eeb9-4315-a403-cf87b73f3b43-kube-api-access-qdpkq\") pod \"postgres-69fb4594cf-j4j25\" (UID: \"e0ff1db7-eeb9-4315-a403-cf87b73f3b43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" Sep 05 06:36:20.416784 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.416764 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0ff1db7-eeb9-4315-a403-cf87b73f3b43-data\") pod \"postgres-69fb4594cf-j4j25\" (UID: \"e0ff1db7-eeb9-4315-a403-cf87b73f3b43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" Sep 05 06:36:20.426528 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.426498 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdpkq\" (UniqueName: \"kubernetes.io/projected/e0ff1db7-eeb9-4315-a403-cf87b73f3b43-kube-api-access-qdpkq\") pod \"postgres-69fb4594cf-j4j25\" (UID: \"e0ff1db7-eeb9-4315-a403-cf87b73f3b43\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" Sep 05 06:36:20.496767 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.496735 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" Sep 05 06:36:20.771861 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:20.771834 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-j4j25"] Sep 05 06:36:20.773592 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:36:20.773561 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ff1db7_eeb9_4315_a403_cf87b73f3b43.slice/crio-ef59da971806efb8dfb5d9b135464ea38a61377e059d156e43832c4460565e9a WatchSource:0}: Error finding container ef59da971806efb8dfb5d9b135464ea38a61377e059d156e43832c4460565e9a: Status 404 returned error can't find the container with id ef59da971806efb8dfb5d9b135464ea38a61377e059d156e43832c4460565e9a Sep 05 06:36:21.151654 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:21.151559 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" event={"ID":"e0ff1db7-eeb9-4315-a403-cf87b73f3b43","Type":"ContainerStarted","Data":"ef59da971806efb8dfb5d9b135464ea38a61377e059d156e43832c4460565e9a"} Sep 05 06:36:26.190347 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:26.190306 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" event={"ID":"e0ff1db7-eeb9-4315-a403-cf87b73f3b43","Type":"ContainerStarted","Data":"91e562c587d8d950d222c8a31675bc0451b1f39b653073b0d1399b518704d77d"} Sep 05 06:36:26.190737 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:26.190427 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" Sep 05 06:36:32.225160 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:32.225129 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" Sep 05 06:36:32.243663 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:32.243601 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-j4j25" podStartSLOduration=7.493361923 podStartE2EDuration="12.243581207s" totalImagesPullingTime="4.750219284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:36:26.208416736 +0000 UTC m=+682.891921645" watchObservedRunningTime="2026-09-05 06:36:32.243581207 +0000 UTC m=+688.927086111" Sep 05 06:36:50.517454 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.517420 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85f7c84d48-78x5c"] Sep 05 06:36:50.539521 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.539468 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85f7c84d48-78x5c"] Sep 05 06:36:50.539710 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.539566 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:50.542771 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.542742 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 05 06:36:50.542903 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.542777 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vlhj9\"" Sep 05 06:36:50.542903 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.542839 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 05 06:36:50.542903 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.542774 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 05 06:36:50.588010 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.587970 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-metrics-certs\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:50.588172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.588029 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-webhook-cert\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:50.588172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.588057 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdtmd\" (UniqueName: \"kubernetes.io/projected/d9ed5692-77f6-459e-8809-bb55f9e1e074-kube-api-access-mdtmd\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:50.688963 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.688922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-metrics-certs\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:50.689158 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.688977 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-webhook-cert\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:50.689158 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.689006 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdtmd\" (UniqueName: \"kubernetes.io/projected/d9ed5692-77f6-459e-8809-bb55f9e1e074-kube-api-access-mdtmd\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:50.689158 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:36:50.689054 2583 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 05 06:36:50.689158 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:36:50.689124 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-metrics-certs podName:d9ed5692-77f6-459e-8809-bb55f9e1e074 nodeName:}" failed. No retries permitted until 2026-09-05 06:36:51.189107404 +0000 UTC m=+707.872612289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-metrics-certs") pod "maas-controller-85f7c84d48-78x5c" (UID: "d9ed5692-77f6-459e-8809-bb55f9e1e074") : secret "maas-controller-metrics-tls" not found Sep 05 06:36:50.689158 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:36:50.689145 2583 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 05 06:36:50.689378 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:36:50.689211 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-webhook-cert podName:d9ed5692-77f6-459e-8809-bb55f9e1e074 nodeName:}" failed. No retries permitted until 2026-09-05 06:36:51.189192494 +0000 UTC m=+707.872697384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-webhook-cert") pod "maas-controller-85f7c84d48-78x5c" (UID: "d9ed5692-77f6-459e-8809-bb55f9e1e074") : secret "maas-controller-webhook-cert" not found Sep 05 06:36:50.697076 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.697052 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55889757f5-rdk4q"] Sep 05 06:36:50.701218 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.701200 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:50.705963 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.705940 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdtmd\" (UniqueName: \"kubernetes.io/projected/d9ed5692-77f6-459e-8809-bb55f9e1e074-kube-api-access-mdtmd\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:50.713675 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.713647 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55889757f5-rdk4q"] Sep 05 06:36:50.790117 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.790029 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwjq5\" (UniqueName: \"kubernetes.io/projected/73644b49-45ee-441e-90c0-b99b829d5499-kube-api-access-kwjq5\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:50.790259 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.790153 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-webhook-cert\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:50.790259 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.790230 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-metrics-certs\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:50.890677 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.890641 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-webhook-cert\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:50.890866 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.890707 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-metrics-certs\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:50.890866 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.890765 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwjq5\" (UniqueName: \"kubernetes.io/projected/73644b49-45ee-441e-90c0-b99b829d5499-kube-api-access-kwjq5\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:50.890866 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:36:50.890794 2583 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 05 06:36:50.890866 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:36:50.890855 2583 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 05 06:36:50.891059 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:36:50.890868 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-webhook-cert podName:73644b49-45ee-441e-90c0-b99b829d5499 nodeName:}" failed. No retries permitted until 2026-09-05 06:36:51.390847211 +0000 UTC m=+708.074352117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-webhook-cert") pod "maas-controller-55889757f5-rdk4q" (UID: "73644b49-45ee-441e-90c0-b99b829d5499") : secret "maas-controller-webhook-cert" not found Sep 05 06:36:50.891059 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:36:50.890926 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-metrics-certs podName:73644b49-45ee-441e-90c0-b99b829d5499 nodeName:}" failed. No retries permitted until 2026-09-05 06:36:51.3909067 +0000 UTC m=+708.074411586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-metrics-certs") pod "maas-controller-55889757f5-rdk4q" (UID: "73644b49-45ee-441e-90c0-b99b829d5499") : secret "maas-controller-metrics-tls" not found Sep 05 06:36:50.900172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:50.900148 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwjq5\" (UniqueName: \"kubernetes.io/projected/73644b49-45ee-441e-90c0-b99b829d5499-kube-api-access-kwjq5\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:51.193460 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.193379 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-metrics-certs\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:51.193460 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.193423 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-webhook-cert\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:51.195926 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.195896 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-metrics-certs\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:51.196043 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.196016 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-webhook-cert\") pod \"maas-controller-85f7c84d48-78x5c\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:51.395247 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.395210 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-metrics-certs\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:51.395414 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.395296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-webhook-cert\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:51.397603 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.397575 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-webhook-cert\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:51.397772 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.397749 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-metrics-certs\") pod \"maas-controller-55889757f5-rdk4q\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:51.452600 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.452519 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:51.626358 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.626323 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:51.725934 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.725905 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85f7c84d48-78x5c"] Sep 05 06:36:51.728165 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:36:51.728137 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ed5692_77f6_459e_8809_bb55f9e1e074.slice/crio-31e3efdbf03175cc720b3873de7c156da3fda8d13a4bfd4bbe95b639c232eade WatchSource:0}: Error finding container 31e3efdbf03175cc720b3873de7c156da3fda8d13a4bfd4bbe95b639c232eade: Status 404 returned error can't find the container with id 31e3efdbf03175cc720b3873de7c156da3fda8d13a4bfd4bbe95b639c232eade Sep 05 06:36:51.899788 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:51.899763 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55889757f5-rdk4q"] Sep 05 06:36:51.901506 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:36:51.901464 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73644b49_45ee_441e_90c0_b99b829d5499.slice/crio-b4c1f938724e13d9061924d374b4421e938b223970ddb69947dcdefd2d46f1b6 WatchSource:0}: Error finding container b4c1f938724e13d9061924d374b4421e938b223970ddb69947dcdefd2d46f1b6: Status 404 returned error can't find the container with id b4c1f938724e13d9061924d374b4421e938b223970ddb69947dcdefd2d46f1b6 Sep 05 06:36:52.334907 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:52.334846 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55889757f5-rdk4q" event={"ID":"73644b49-45ee-441e-90c0-b99b829d5499","Type":"ContainerStarted","Data":"b4c1f938724e13d9061924d374b4421e938b223970ddb69947dcdefd2d46f1b6"} Sep 05 06:36:52.336210 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:52.336149 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85f7c84d48-78x5c" event={"ID":"d9ed5692-77f6-459e-8809-bb55f9e1e074","Type":"ContainerStarted","Data":"31e3efdbf03175cc720b3873de7c156da3fda8d13a4bfd4bbe95b639c232eade"} Sep 05 06:36:55.354177 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:55.354140 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55889757f5-rdk4q" event={"ID":"73644b49-45ee-441e-90c0-b99b829d5499","Type":"ContainerStarted","Data":"d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791"} Sep 05 06:36:55.354668 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:55.354334 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:36:55.355846 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:55.355818 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85f7c84d48-78x5c" event={"ID":"d9ed5692-77f6-459e-8809-bb55f9e1e074","Type":"ContainerStarted","Data":"1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf"} Sep 05 06:36:55.355979 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:55.355908 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:36:55.377303 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:55.377247 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55889757f5-rdk4q" podStartSLOduration=2.8962645670000002 podStartE2EDuration="5.377232708s" totalImagesPullingTime="2.480968141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:36:55.375623133 +0000 UTC m=+712.059128041" watchObservedRunningTime="2026-09-05 06:36:55.377232708 +0000 UTC m=+712.060737615" Sep 05 06:36:55.400470 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:36:55.400424 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85f7c84d48-78x5c" podStartSLOduration=2.75263362 podStartE2EDuration="5.40041083s" totalImagesPullingTime="2.64777721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:36:55.398614357 +0000 UTC m=+712.082119266" watchObservedRunningTime="2026-09-05 06:36:55.40041083 +0000 UTC m=+712.083915736" Sep 05 06:37:06.364443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.364358 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:37:06.364868 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.364528 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:37:06.432574 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.432539 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85f7c84d48-78x5c"] Sep 05 06:37:06.432797 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.432768 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-85f7c84d48-78x5c" podUID="d9ed5692-77f6-459e-8809-bb55f9e1e074" containerName="manager" containerID="cri-o://1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf" gracePeriod=30 Sep 05 06:37:06.679128 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.679104 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:37:06.729523 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.729458 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-metrics-certs\") pod \"d9ed5692-77f6-459e-8809-bb55f9e1e074\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " Sep 05 06:37:06.729523 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.729521 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-webhook-cert\") pod \"d9ed5692-77f6-459e-8809-bb55f9e1e074\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " Sep 05 06:37:06.729767 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.729593 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdtmd\" (UniqueName: \"kubernetes.io/projected/d9ed5692-77f6-459e-8809-bb55f9e1e074-kube-api-access-mdtmd\") pod \"d9ed5692-77f6-459e-8809-bb55f9e1e074\" (UID: \"d9ed5692-77f6-459e-8809-bb55f9e1e074\") " Sep 05 06:37:06.731713 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.731679 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d9ed5692-77f6-459e-8809-bb55f9e1e074" (UID: "d9ed5692-77f6-459e-8809-bb55f9e1e074"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:37:06.731713 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.731691 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d9ed5692-77f6-459e-8809-bb55f9e1e074" (UID: "d9ed5692-77f6-459e-8809-bb55f9e1e074"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:37:06.731847 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.731738 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9ed5692-77f6-459e-8809-bb55f9e1e074-kube-api-access-mdtmd" (OuterVolumeSpecName: "kube-api-access-mdtmd") pod "d9ed5692-77f6-459e-8809-bb55f9e1e074" (UID: "d9ed5692-77f6-459e-8809-bb55f9e1e074"). InnerVolumeSpecName "kube-api-access-mdtmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:37:06.830504 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.830459 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:37:06.830504 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.830496 2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d9ed5692-77f6-459e-8809-bb55f9e1e074-webhook-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:37:06.830504 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:06.830518 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdtmd\" (UniqueName: \"kubernetes.io/projected/d9ed5692-77f6-459e-8809-bb55f9e1e074-kube-api-access-mdtmd\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:37:07.433471 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:07.433434 2583 generic.go:356] "Generic (PLEG): container finished" podID="d9ed5692-77f6-459e-8809-bb55f9e1e074" containerID="1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf" exitCode=0 Sep 05 06:37:07.433919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:07.433503 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85f7c84d48-78x5c" Sep 05 06:37:07.433919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:07.433509 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85f7c84d48-78x5c" event={"ID":"d9ed5692-77f6-459e-8809-bb55f9e1e074","Type":"ContainerDied","Data":"1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf"} Sep 05 06:37:07.433919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:07.433570 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85f7c84d48-78x5c" event={"ID":"d9ed5692-77f6-459e-8809-bb55f9e1e074","Type":"ContainerDied","Data":"31e3efdbf03175cc720b3873de7c156da3fda8d13a4bfd4bbe95b639c232eade"} Sep 05 06:37:07.433919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:07.433593 2583 scope.go:117] "RemoveContainer" containerID="1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf" Sep 05 06:37:07.443255 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:07.443240 2583 scope.go:117] "RemoveContainer" containerID="1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf" Sep 05 06:37:07.443573 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:37:07.443548 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf\": container with ID starting with 1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf not found: ID does not exist" containerID="1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf" Sep 05 06:37:07.443649 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:07.443578 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf"} err="failed to get container status \"1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf\": rpc error: code = NotFound desc = could not find container \"1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf\": container with ID starting with 1e8a7ab7e3f81c1d4172d250f42b2a2e32de8f6dc3d6df7b0cda14776d6d8ccf not found: ID does not exist" Sep 05 06:37:07.463050 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:07.463022 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85f7c84d48-78x5c"] Sep 05 06:37:07.465231 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:07.465212 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-85f7c84d48-78x5c"] Sep 05 06:37:07.822834 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:07.822801 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9ed5692-77f6-459e-8809-bb55f9e1e074" path="/var/lib/kubelet/pods/d9ed5692-77f6-459e-8809-bb55f9e1e074/volumes" Sep 05 06:37:26.535965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.535930 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz"] Sep 05 06:37:26.537094 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.537072 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9ed5692-77f6-459e-8809-bb55f9e1e074" containerName="manager" Sep 05 06:37:26.537184 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.537098 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9ed5692-77f6-459e-8809-bb55f9e1e074" containerName="manager" Sep 05 06:37:26.537274 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.537248 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9ed5692-77f6-459e-8809-bb55f9e1e074" containerName="manager" Sep 05 06:37:26.550302 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.550269 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.553211 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.553189 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 05 06:37:26.553361 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.553341 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hpktq\"" Sep 05 06:37:26.553540 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.553281 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 05 06:37:26.553666 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.553640 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz"] Sep 05 06:37:26.696956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.696924 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggh2g\" (UniqueName: \"kubernetes.io/projected/5ec03923-58c6-4931-8f31-e186f366f284-kube-api-access-ggh2g\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.697119 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.696970 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-maas-api-tls\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.697119 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.697036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec03923-58c6-4931-8f31-e186f366f284-openshift-service-ca\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.697119 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.697054 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-metrics-certs\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.798042 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.797969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggh2g\" (UniqueName: \"kubernetes.io/projected/5ec03923-58c6-4931-8f31-e186f366f284-kube-api-access-ggh2g\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.798042 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.798015 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-maas-api-tls\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.798261 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.798121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec03923-58c6-4931-8f31-e186f366f284-openshift-service-ca\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.798261 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.798145 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-metrics-certs\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.798903 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.798879 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec03923-58c6-4931-8f31-e186f366f284-openshift-service-ca\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.800533 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.800507 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-maas-api-tls\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.800667 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.800644 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-metrics-certs\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.806266 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.806244 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggh2g\" (UniqueName: \"kubernetes.io/projected/5ec03923-58c6-4931-8f31-e186f366f284-kube-api-access-ggh2g\") pod \"maas-api-644985f7bf-hb5hz\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:26.863276 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:26.863239 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:27.150220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:27.150190 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz"] Sep 05 06:37:27.151211 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:37:27.151185 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ec03923_58c6_4931_8f31_e186f366f284.slice/crio-9da3f35b6c91797b3a224a05894e6f98356f2c7582f04df61a013180cd358f4d WatchSource:0}: Error finding container 9da3f35b6c91797b3a224a05894e6f98356f2c7582f04df61a013180cd358f4d: Status 404 returned error can't find the container with id 9da3f35b6c91797b3a224a05894e6f98356f2c7582f04df61a013180cd358f4d Sep 05 06:37:27.152625 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:27.152606 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 05 06:37:27.549735 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:27.549683 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" event={"ID":"5ec03923-58c6-4931-8f31-e186f366f284","Type":"ContainerStarted","Data":"9da3f35b6c91797b3a224a05894e6f98356f2c7582f04df61a013180cd358f4d"} Sep 05 06:37:29.563724 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:29.563685 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" event={"ID":"5ec03923-58c6-4931-8f31-e186f366f284","Type":"ContainerStarted","Data":"879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc"} Sep 05 06:37:29.564153 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:29.563752 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:29.584686 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:29.584636 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" podStartSLOduration=2.107452417 podStartE2EDuration="3.584620763s" totalImagesPullingTime="1.477168346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:37:29.582083129 +0000 UTC m=+746.265588039" watchObservedRunningTime="2026-09-05 06:37:29.584620763 +0000 UTC m=+746.268125697" Sep 05 06:37:35.575353 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:35.575322 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 06:37:48.675241 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.675204 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff"] Sep 05 06:37:48.679107 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.679086 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.682037 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.682008 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 05 06:37:48.682819 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.682795 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 05 06:37:48.682957 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.682940 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 05 06:37:48.683225 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.683204 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wkllk\"" Sep 05 06:37:48.690992 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.690967 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff"] Sep 05 06:37:48.708165 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.708140 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.708375 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.708196 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.708375 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.708290 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.708375 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.708318 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mrc7\" (UniqueName: \"kubernetes.io/projected/569eecc6-3781-4b66-b3ce-9b119ee1a87e-kube-api-access-5mrc7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.708375 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.708339 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569eecc6-3781-4b66-b3ce-9b119ee1a87e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.708375 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.708363 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.809292 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.809250 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.809463 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.809354 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.809463 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.809443 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.809633 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.809508 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mrc7\" (UniqueName: \"kubernetes.io/projected/569eecc6-3781-4b66-b3ce-9b119ee1a87e-kube-api-access-5mrc7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.809633 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.809541 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569eecc6-3781-4b66-b3ce-9b119ee1a87e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.809633 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.809569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.809860 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.809836 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.809951 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.809889 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.809951 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.809925 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.811536 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.811517 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569eecc6-3781-4b66-b3ce-9b119ee1a87e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.811949 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.811933 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569eecc6-3781-4b66-b3ce-9b119ee1a87e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.819467 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.819442 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mrc7\" (UniqueName: \"kubernetes.io/projected/569eecc6-3781-4b66-b3ce-9b119ee1a87e-kube-api-access-5mrc7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff\" (UID: \"569eecc6-3781-4b66-b3ce-9b119ee1a87e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:48.990157 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:48.990126 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:49.267600 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:49.267570 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff"] Sep 05 06:37:49.269254 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:37:49.269231 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod569eecc6_3781_4b66_b3ce_9b119ee1a87e.slice/crio-c8c14c760eba3fc1e432f33f31a4fca2dfe5a0f03db2ea29c1666a9447557080 WatchSource:0}: Error finding container c8c14c760eba3fc1e432f33f31a4fca2dfe5a0f03db2ea29c1666a9447557080: Status 404 returned error can't find the container with id c8c14c760eba3fc1e432f33f31a4fca2dfe5a0f03db2ea29c1666a9447557080 Sep 05 06:37:49.679721 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:49.679624 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" event={"ID":"569eecc6-3781-4b66-b3ce-9b119ee1a87e","Type":"ContainerStarted","Data":"c8c14c760eba3fc1e432f33f31a4fca2dfe5a0f03db2ea29c1666a9447557080"} Sep 05 06:37:50.687883 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:50.687846 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" event={"ID":"569eecc6-3781-4b66-b3ce-9b119ee1a87e","Type":"ContainerStarted","Data":"1e6596c16af0df7d33b961c935d1549e8b8a1b54cf7b0faa6392e201047af748"} Sep 05 06:37:50.709541 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:50.709464 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" podStartSLOduration=1.751702724 podStartE2EDuration="2.709446128s" totalImagesPullingTime="957.743404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:37:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:37:50.708466884 +0000 UTC m=+767.391971794" watchObservedRunningTime="2026-09-05 06:37:50.709446128 +0000 UTC m=+767.392951035" Sep 05 06:37:58.990439 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:58.990395 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:58.990901 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:58.990469 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:59.003155 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:59.003129 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:37:59.747663 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:37:59.747633 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff" Sep 05 06:38:01.658809 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.658773 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg"] Sep 05 06:38:01.662443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.662423 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.665367 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.665345 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 05 06:38:01.673053 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.673027 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg"] Sep 05 06:38:01.730805 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.730776 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lzdj\" (UniqueName: \"kubernetes.io/projected/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-kube-api-access-8lzdj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.730956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.730822 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.730956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.730861 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.730956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.730887 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.730956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.730908 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.731108 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.730992 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.831689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.831655 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.831844 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.831696 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.831844 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.831723 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.831934 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.831833 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.831934 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.831917 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.832027 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.832014 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lzdj\" (UniqueName: \"kubernetes.io/projected/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-kube-api-access-8lzdj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.832106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.832082 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.832305 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.832285 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.832430 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.832343 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.834322 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.834296 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.834516 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.834495 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.840240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.840215 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lzdj\" (UniqueName: \"kubernetes.io/projected/3f0ca2da-42ed-4b6e-b663-9a14527a46a4-kube-api-access-8lzdj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mxfgg\" (UID: \"3f0ca2da-42ed-4b6e-b663-9a14527a46a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:01.973671 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:01.973588 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:02.250992 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:02.250906 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg"] Sep 05 06:38:02.253414 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:38:02.253386 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0ca2da_42ed_4b6e_b663_9a14527a46a4.slice/crio-86d96c63ae02ba436663d32bebb570661179f3fc9ebdb416206fa22e144ef518 WatchSource:0}: Error finding container 86d96c63ae02ba436663d32bebb570661179f3fc9ebdb416206fa22e144ef518: Status 404 returned error can't find the container with id 86d96c63ae02ba436663d32bebb570661179f3fc9ebdb416206fa22e144ef518 Sep 05 06:38:02.755482 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:02.755447 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" event={"ID":"3f0ca2da-42ed-4b6e-b663-9a14527a46a4","Type":"ContainerStarted","Data":"2739740ff06de706fd27eb61615580272fb323fc03d6fdb84ad1603886fa2bff"} Sep 05 06:38:02.755855 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:02.755510 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" event={"ID":"3f0ca2da-42ed-4b6e-b663-9a14527a46a4","Type":"ContainerStarted","Data":"86d96c63ae02ba436663d32bebb570661179f3fc9ebdb416206fa22e144ef518"} Sep 05 06:38:02.775519 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:02.775447 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" podStartSLOduration=1.603286223 podStartE2EDuration="1.775432089s" totalImagesPullingTime="172.145866ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:38:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:38:02.77485009 +0000 UTC m=+779.458354998" watchObservedRunningTime="2026-09-05 06:38:02.775432089 +0000 UTC m=+779.458937012" Sep 05 06:38:11.974431 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:11.974390 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:11.974896 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:11.974448 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:11.986705 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:11.986682 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:38:12.821205 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:38:12.821173 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mxfgg" Sep 05 06:40:03.816445 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:03.816358 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:40:03.819114 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:03.819087 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:40:03.833352 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:03.833326 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:40:03.835345 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:03.835323 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:40:19.702997 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.702961 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf"] Sep 05 06:40:19.707145 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.707124 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.709690 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.709665 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6efc3a-serving-cert\"" Sep 05 06:40:19.722131 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.722102 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf"] Sep 05 06:40:19.772829 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.772795 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-maas-api-tls\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.772963 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.772844 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrlwn\" (UniqueName: \"kubernetes.io/projected/84a075fb-d230-4596-b8fe-9c1458e823a1-kube-api-access-xrlwn\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.772963 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.772947 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-metrics-certs\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.773039 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.773014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a075fb-d230-4596-b8fe-9c1458e823a1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.873837 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.873803 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a075fb-d230-4596-b8fe-9c1458e823a1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.873992 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.873971 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-maas-api-tls\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.874043 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.874005 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrlwn\" (UniqueName: \"kubernetes.io/projected/84a075fb-d230-4596-b8fe-9c1458e823a1-kube-api-access-xrlwn\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.874081 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.874056 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-metrics-certs\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.874135 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:40:19.874115 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-serving-cert: secret "maas-api-e2e-inf-a-6efc3a-serving-cert" not found Sep 05 06:40:19.874201 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:40:19.874191 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-maas-api-tls podName:84a075fb-d230-4596-b8fe-9c1458e823a1 nodeName:}" failed. No retries permitted until 2026-09-05 06:40:20.374171015 +0000 UTC m=+917.057675917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-maas-api-tls") pod "maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" (UID: "84a075fb-d230-4596-b8fe-9c1458e823a1") : secret "maas-api-e2e-inf-a-6efc3a-serving-cert" not found Sep 05 06:40:19.874630 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.874607 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a075fb-d230-4596-b8fe-9c1458e823a1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.876450 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.876429 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-metrics-certs\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:19.883899 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:19.883875 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrlwn\" (UniqueName: \"kubernetes.io/projected/84a075fb-d230-4596-b8fe-9c1458e823a1-kube-api-access-xrlwn\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:20.378378 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:20.378341 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-maas-api-tls\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:20.380813 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:20.380787 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-maas-api-tls\") pod \"maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:20.618177 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:20.618140 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:20.898529 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:20.898497 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf"] Sep 05 06:40:20.899381 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:40:20.899356 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a075fb_d230_4596_b8fe_9c1458e823a1.slice/crio-a621a233f1732a2859562018044988b63bd97e9abe5cf22e70e881407398ab6e WatchSource:0}: Error finding container a621a233f1732a2859562018044988b63bd97e9abe5cf22e70e881407398ab6e: Status 404 returned error can't find the container with id a621a233f1732a2859562018044988b63bd97e9abe5cf22e70e881407398ab6e Sep 05 06:40:21.552576 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:21.552539 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" event={"ID":"84a075fb-d230-4596-b8fe-9c1458e823a1","Type":"ContainerStarted","Data":"01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c"} Sep 05 06:40:21.552757 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:21.552601 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" event={"ID":"84a075fb-d230-4596-b8fe-9c1458e823a1","Type":"ContainerStarted","Data":"a621a233f1732a2859562018044988b63bd97e9abe5cf22e70e881407398ab6e"} Sep 05 06:40:21.552830 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:21.552765 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:21.575117 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:21.575068 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" podStartSLOduration=2.019752888 podStartE2EDuration="2.575052799s" totalImagesPullingTime="555.299911ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:21.572734281 +0000 UTC m=+918.256239188" watchObservedRunningTime="2026-09-05 06:40:21.575052799 +0000 UTC m=+918.258557705" Sep 05 06:40:27.277300 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.277256 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh"] Sep 05 06:40:27.281639 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.281617 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.284297 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.284272 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-faf2bf41-serving-cert\"" Sep 05 06:40:27.301115 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.301088 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh"] Sep 05 06:40:27.342639 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.342595 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-maas-api-tls\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.342639 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.342643 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff58d44-61cf-4294-9bb2-1b955058a988-openshift-service-ca\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.342822 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.342745 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-metrics-certs\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.342863 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.342848 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8q9k\" (UniqueName: \"kubernetes.io/projected/eff58d44-61cf-4294-9bb2-1b955058a988-kube-api-access-l8q9k\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.444051 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.444020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8q9k\" (UniqueName: \"kubernetes.io/projected/eff58d44-61cf-4294-9bb2-1b955058a988-kube-api-access-l8q9k\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.444219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.444069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-maas-api-tls\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.444219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.444112 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff58d44-61cf-4294-9bb2-1b955058a988-openshift-service-ca\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.444219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.444150 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-metrics-certs\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.444821 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.444797 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff58d44-61cf-4294-9bb2-1b955058a988-openshift-service-ca\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.446707 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.446681 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-metrics-certs\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.446793 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.446716 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-maas-api-tls\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.453869 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.453850 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8q9k\" (UniqueName: \"kubernetes.io/projected/eff58d44-61cf-4294-9bb2-1b955058a988-kube-api-access-l8q9k\") pod \"maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.564781 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.564709 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:40:27.594151 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.594114 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:27.889141 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:27.889111 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh"] Sep 05 06:40:27.890868 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:40:27.890839 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff58d44_61cf_4294_9bb2_1b955058a988.slice/crio-9a9e4ea44623200fbadd242a5e57b04c63641b861cdc9228c09d825943bdea47 WatchSource:0}: Error finding container 9a9e4ea44623200fbadd242a5e57b04c63641b861cdc9228c09d825943bdea47: Status 404 returned error can't find the container with id 9a9e4ea44623200fbadd242a5e57b04c63641b861cdc9228c09d825943bdea47 Sep 05 06:40:28.595621 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:28.595520 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" event={"ID":"eff58d44-61cf-4294-9bb2-1b955058a988","Type":"ContainerStarted","Data":"ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7"} Sep 05 06:40:28.595621 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:28.595588 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" event={"ID":"eff58d44-61cf-4294-9bb2-1b955058a988","Type":"ContainerStarted","Data":"9a9e4ea44623200fbadd242a5e57b04c63641b861cdc9228c09d825943bdea47"} Sep 05 06:40:28.596027 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:28.595725 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:28.623211 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:28.623166 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" podStartSLOduration=1.190588399 podStartE2EDuration="1.623149744s" totalImagesPullingTime="432.561345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:28.622190609 +0000 UTC m=+925.305695517" watchObservedRunningTime="2026-09-05 06:40:28.623149744 +0000 UTC m=+925.306654650" Sep 05 06:40:29.319656 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.319618 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9"] Sep 05 06:40:29.323561 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.323538 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.326767 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.326747 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-e04a57-serving-cert\"" Sep 05 06:40:29.349535 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.349511 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9"] Sep 05 06:40:29.463805 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.463773 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-maas-api-tls\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.463805 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.463808 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfsqx\" (UniqueName: \"kubernetes.io/projected/d64cd300-a3e9-42ff-80f2-9ae248badb05-kube-api-access-jfsqx\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.464011 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.463904 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64cd300-a3e9-42ff-80f2-9ae248badb05-openshift-service-ca\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.464057 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.463984 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-metrics-certs\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.565425 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.565395 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-metrics-certs\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.565620 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.565456 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-maas-api-tls\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.565620 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.565496 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfsqx\" (UniqueName: \"kubernetes.io/projected/d64cd300-a3e9-42ff-80f2-9ae248badb05-kube-api-access-jfsqx\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.565620 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.565570 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64cd300-a3e9-42ff-80f2-9ae248badb05-openshift-service-ca\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.565620 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:40:29.565580 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-serving-cert: secret "maas-api-e2e-autores-e04a57-serving-cert" not found Sep 05 06:40:29.565818 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:40:29.565643 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-maas-api-tls podName:d64cd300-a3e9-42ff-80f2-9ae248badb05 nodeName:}" failed. No retries permitted until 2026-09-05 06:40:30.065623156 +0000 UTC m=+926.749128044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-maas-api-tls") pod "maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" (UID: "d64cd300-a3e9-42ff-80f2-9ae248badb05") : secret "maas-api-e2e-autores-e04a57-serving-cert" not found Sep 05 06:40:29.566291 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.566262 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64cd300-a3e9-42ff-80f2-9ae248badb05-openshift-service-ca\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.567913 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.567890 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-metrics-certs\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:29.577551 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:29.577500 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfsqx\" (UniqueName: \"kubernetes.io/projected/d64cd300-a3e9-42ff-80f2-9ae248badb05-kube-api-access-jfsqx\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:30.070673 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:30.070636 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-maas-api-tls\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:30.073257 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:30.073211 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-maas-api-tls\") pod \"maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:30.235795 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:30.235758 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:30.519342 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:30.519304 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9"] Sep 05 06:40:30.608664 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:30.608625 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" event={"ID":"d64cd300-a3e9-42ff-80f2-9ae248badb05","Type":"ContainerStarted","Data":"b77bbbd80a0ca71d5d3885ad8ec5904376727362fe6cdcace1120e200f615423"} Sep 05 06:40:31.616347 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:31.616307 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" event={"ID":"d64cd300-a3e9-42ff-80f2-9ae248badb05","Type":"ContainerStarted","Data":"17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c"} Sep 05 06:40:31.616803 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:31.616372 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:31.639458 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:31.639410 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" podStartSLOduration=2.258120516 podStartE2EDuration="2.639395053s" totalImagesPullingTime="381.274537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:31.635817149 +0000 UTC m=+928.319322056" watchObservedRunningTime="2026-09-05 06:40:31.639395053 +0000 UTC m=+928.322899959" Sep 05 06:40:34.606447 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:34.606414 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:40:37.628056 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:37.628024 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:40:48.835188 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.835155 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj"] Sep 05 06:40:48.839015 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.838992 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.841327 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.841305 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-5d5f17-serving-cert\"" Sep 05 06:40:48.847787 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.847522 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-metrics-certs\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.847787 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.847590 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-maas-api-tls\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.847787 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.847748 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bab9b006-130c-47f1-8d5e-297452e89116-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.848014 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.847800 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kf86\" (UniqueName: \"kubernetes.io/projected/bab9b006-130c-47f1-8d5e-297452e89116-kube-api-access-7kf86\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.862918 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.862888 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj"] Sep 05 06:40:48.948398 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.948360 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bab9b006-130c-47f1-8d5e-297452e89116-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.948625 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.948407 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kf86\" (UniqueName: \"kubernetes.io/projected/bab9b006-130c-47f1-8d5e-297452e89116-kube-api-access-7kf86\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.948625 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.948500 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-metrics-certs\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.948625 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.948545 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-maas-api-tls\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.948997 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.948972 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bab9b006-130c-47f1-8d5e-297452e89116-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.951027 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.951002 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-metrics-certs\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.951130 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.951026 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-maas-api-tls\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:48.965571 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:48.965538 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kf86\" (UniqueName: \"kubernetes.io/projected/bab9b006-130c-47f1-8d5e-297452e89116-kube-api-access-7kf86\") pod \"maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:49.153395 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.153302 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:49.434116 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.434090 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj"] Sep 05 06:40:49.436704 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:40:49.436677 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab9b006_130c_47f1_8d5e_297452e89116.slice/crio-b6981d539346791390be017f507dd6fc6fa2fba30d0f8e1629d5071a93cc9aaf WatchSource:0}: Error finding container b6981d539346791390be017f507dd6fc6fa2fba30d0f8e1629d5071a93cc9aaf: Status 404 returned error can't find the container with id b6981d539346791390be017f507dd6fc6fa2fba30d0f8e1629d5071a93cc9aaf Sep 05 06:40:49.682240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.682200 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb"] Sep 05 06:40:49.687324 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.687269 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:40:49.689821 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.689795 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-5d5f17\"" Sep 05 06:40:49.702745 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.702721 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb"] Sep 05 06:40:49.724346 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.724303 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" event={"ID":"bab9b006-130c-47f1-8d5e-297452e89116","Type":"ContainerStarted","Data":"b6981d539346791390be017f507dd6fc6fa2fba30d0f8e1629d5071a93cc9aaf"} Sep 05 06:40:49.755145 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.755117 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64xtd\" (UniqueName: \"kubernetes.io/projected/cc4aad14-143f-4276-86ce-8f9e2470bf74-kube-api-access-64xtd\") pod \"payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb\" (UID: \"cc4aad14-143f-4276-86ce-8f9e2470bf74\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:40:49.755269 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.755172 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc4aad14-143f-4276-86ce-8f9e2470bf74-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb\" (UID: \"cc4aad14-143f-4276-86ce-8f9e2470bf74\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:40:49.856362 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.856324 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64xtd\" (UniqueName: \"kubernetes.io/projected/cc4aad14-143f-4276-86ce-8f9e2470bf74-kube-api-access-64xtd\") pod \"payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb\" (UID: \"cc4aad14-143f-4276-86ce-8f9e2470bf74\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:40:49.856780 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.856420 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc4aad14-143f-4276-86ce-8f9e2470bf74-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb\" (UID: \"cc4aad14-143f-4276-86ce-8f9e2470bf74\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:40:49.857200 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.857178 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc4aad14-143f-4276-86ce-8f9e2470bf74-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb\" (UID: \"cc4aad14-143f-4276-86ce-8f9e2470bf74\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:40:49.868027 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.867996 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64xtd\" (UniqueName: \"kubernetes.io/projected/cc4aad14-143f-4276-86ce-8f9e2470bf74-kube-api-access-64xtd\") pod \"payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb\" (UID: \"cc4aad14-143f-4276-86ce-8f9e2470bf74\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:40:49.999336 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:49.999304 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:40:50.279554 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:50.279528 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb"] Sep 05 06:40:50.281255 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:40:50.281223 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc4aad14_143f_4276_86ce_8f9e2470bf74.slice/crio-28e8efefae4b3bc908c3302abc308050d599e9735d146b245e55fabda57223c6 WatchSource:0}: Error finding container 28e8efefae4b3bc908c3302abc308050d599e9735d146b245e55fabda57223c6: Status 404 returned error can't find the container with id 28e8efefae4b3bc908c3302abc308050d599e9735d146b245e55fabda57223c6 Sep 05 06:40:50.731849 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:50.731811 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" event={"ID":"bab9b006-130c-47f1-8d5e-297452e89116","Type":"ContainerStarted","Data":"4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b"} Sep 05 06:40:50.732117 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:50.731996 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:50.733146 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:50.733122 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" event={"ID":"cc4aad14-143f-4276-86ce-8f9e2470bf74","Type":"ContainerStarted","Data":"28e8efefae4b3bc908c3302abc308050d599e9735d146b245e55fabda57223c6"} Sep 05 06:40:50.758661 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:50.758611 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" podStartSLOduration=2.47275439 podStartE2EDuration="2.75859558s" totalImagesPullingTime="285.84119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:50.757294975 +0000 UTC m=+947.440799882" watchObservedRunningTime="2026-09-05 06:40:50.75859558 +0000 UTC m=+947.442100487" Sep 05 06:40:52.748608 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:52.748570 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" event={"ID":"cc4aad14-143f-4276-86ce-8f9e2470bf74","Type":"ContainerStarted","Data":"3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459"} Sep 05 06:40:52.749086 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:52.748707 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:40:56.747625 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:56.747590 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:40:56.770974 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:40:56.770926 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" podStartSLOduration=6.069820842 podStartE2EDuration="7.770910928s" totalImagesPullingTime="1.701090086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:52.789561559 +0000 UTC m=+949.473066465" watchObservedRunningTime="2026-09-05 06:40:56.770910928 +0000 UTC m=+953.454415876" Sep 05 06:41:03.763993 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:03.763963 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:41:08.020283 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.019711 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh"] Sep 05 06:41:08.020283 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.019978 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" podUID="eff58d44-61cf-4294-9bb2-1b955058a988" containerName="maas-api" containerID="cri-o://ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7" gracePeriod=30 Sep 05 06:41:08.279552 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.279471 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:41:08.329642 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.329593 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff58d44-61cf-4294-9bb2-1b955058a988-openshift-service-ca\") pod \"eff58d44-61cf-4294-9bb2-1b955058a988\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " Sep 05 06:41:08.329817 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.329771 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8q9k\" (UniqueName: \"kubernetes.io/projected/eff58d44-61cf-4294-9bb2-1b955058a988-kube-api-access-l8q9k\") pod \"eff58d44-61cf-4294-9bb2-1b955058a988\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " Sep 05 06:41:08.329892 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.329821 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-maas-api-tls\") pod \"eff58d44-61cf-4294-9bb2-1b955058a988\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " Sep 05 06:41:08.329892 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.329857 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-metrics-certs\") pod \"eff58d44-61cf-4294-9bb2-1b955058a988\" (UID: \"eff58d44-61cf-4294-9bb2-1b955058a988\") " Sep 05 06:41:08.330005 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.329972 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eff58d44-61cf-4294-9bb2-1b955058a988-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eff58d44-61cf-4294-9bb2-1b955058a988" (UID: "eff58d44-61cf-4294-9bb2-1b955058a988"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:41:08.330201 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.330182 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff58d44-61cf-4294-9bb2-1b955058a988-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:08.331947 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.331920 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eff58d44-61cf-4294-9bb2-1b955058a988" (UID: "eff58d44-61cf-4294-9bb2-1b955058a988"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:41:08.331947 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.331939 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eff58d44-61cf-4294-9bb2-1b955058a988" (UID: "eff58d44-61cf-4294-9bb2-1b955058a988"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:41:08.332086 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.332002 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff58d44-61cf-4294-9bb2-1b955058a988-kube-api-access-l8q9k" (OuterVolumeSpecName: "kube-api-access-l8q9k") pod "eff58d44-61cf-4294-9bb2-1b955058a988" (UID: "eff58d44-61cf-4294-9bb2-1b955058a988"). InnerVolumeSpecName "kube-api-access-l8q9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:41:08.431783 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.431747 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:08.431783 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.431784 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8q9k\" (UniqueName: \"kubernetes.io/projected/eff58d44-61cf-4294-9bb2-1b955058a988-kube-api-access-l8q9k\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:08.432019 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.431802 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eff58d44-61cf-4294-9bb2-1b955058a988-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:08.853097 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.853063 2583 generic.go:356] "Generic (PLEG): container finished" podID="eff58d44-61cf-4294-9bb2-1b955058a988" containerID="ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7" exitCode=0 Sep 05 06:41:08.853276 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.853108 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" Sep 05 06:41:08.853276 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.853149 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" event={"ID":"eff58d44-61cf-4294-9bb2-1b955058a988","Type":"ContainerDied","Data":"ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7"} Sep 05 06:41:08.853276 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.853201 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh" event={"ID":"eff58d44-61cf-4294-9bb2-1b955058a988","Type":"ContainerDied","Data":"9a9e4ea44623200fbadd242a5e57b04c63641b861cdc9228c09d825943bdea47"} Sep 05 06:41:08.853276 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.853227 2583 scope.go:117] "RemoveContainer" containerID="ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7" Sep 05 06:41:08.863646 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.863628 2583 scope.go:117] "RemoveContainer" containerID="ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7" Sep 05 06:41:08.863913 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:41:08.863897 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7\": container with ID starting with ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7 not found: ID does not exist" containerID="ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7" Sep 05 06:41:08.863976 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.863922 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7"} err="failed to get container status \"ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7\": rpc error: code = NotFound desc = could not find container \"ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7\": container with ID starting with ddf41765cd07f56529c774826d8672872448dea291603cd2fc0181d6ee6db5c7 not found: ID does not exist" Sep 05 06:41:08.891809 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.891778 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh"] Sep 05 06:41:08.905128 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:08.905100 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-faf2bf41-b95bbbf5d-jwpbh"] Sep 05 06:41:09.549668 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.549640 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz"] Sep 05 06:41:09.550146 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.550126 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff58d44-61cf-4294-9bb2-1b955058a988" containerName="maas-api" Sep 05 06:41:09.550146 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.550144 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff58d44-61cf-4294-9bb2-1b955058a988" containerName="maas-api" Sep 05 06:41:09.550310 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.550234 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff58d44-61cf-4294-9bb2-1b955058a988" containerName="maas-api" Sep 05 06:41:09.559305 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.559032 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.562641 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.562619 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ea7120c2-gw-openshift-default-dockercfg-2rwms\"" Sep 05 06:41:09.593669 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.593643 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz"] Sep 05 06:41:09.643534 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.643499 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59b55d7e-3fd4-41db-be04-452970ecab05-istiod-ca-cert\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.643534 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.643537 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59b55d7e-3fd4-41db-be04-452970ecab05-istio-podinfo\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.643733 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.643655 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-istio-token\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.643733 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.643705 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-credential-socket\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.643810 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.643763 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-envoy\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.643810 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.643794 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-socket\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.643872 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.643833 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-data\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.643922 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.643869 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-certs\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.643966 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.643918 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpbgn\" (UniqueName: \"kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-kube-api-access-bpbgn\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745033 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.744998 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-istio-token\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745216 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745051 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-credential-socket\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745216 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745078 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-envoy\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745216 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745097 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-socket\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745216 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745129 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-data\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745216 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745145 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-certs\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745216 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745168 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpbgn\" (UniqueName: \"kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-kube-api-access-bpbgn\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745216 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745200 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59b55d7e-3fd4-41db-be04-452970ecab05-istiod-ca-cert\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745602 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745229 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59b55d7e-3fd4-41db-be04-452970ecab05-istio-podinfo\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745602 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745453 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-credential-socket\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745684 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745660 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-data\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745722 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745684 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-certs\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.745932 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745907 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-socket\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.746069 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.745965 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59b55d7e-3fd4-41db-be04-452970ecab05-istiod-ca-cert\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.747836 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.747811 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-envoy\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.747931 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.747888 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59b55d7e-3fd4-41db-be04-452970ecab05-istio-podinfo\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.760369 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.760337 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-istio-token\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.760450 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.760388 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpbgn\" (UniqueName: \"kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-kube-api-access-bpbgn\") pod \"e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:09.824771 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.824669 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff58d44-61cf-4294-9bb2-1b955058a988" path="/var/lib/kubelet/pods/eff58d44-61cf-4294-9bb2-1b955058a988/volumes" Sep 05 06:41:09.871442 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:09.871414 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:10.156897 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:10.156871 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz"] Sep 05 06:41:10.158265 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:41:10.158240 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59b55d7e_3fd4_41db_be04_452970ecab05.slice/crio-beb122f290e621e41931f71f0824e293b7863063022de5a56dc27b2fde378c7b WatchSource:0}: Error finding container beb122f290e621e41931f71f0824e293b7863063022de5a56dc27b2fde378c7b: Status 404 returned error can't find the container with id beb122f290e621e41931f71f0824e293b7863063022de5a56dc27b2fde378c7b Sep 05 06:41:10.867625 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:10.867592 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" event={"ID":"59b55d7e-3fd4-41db-be04-452970ecab05","Type":"ContainerStarted","Data":"beb122f290e621e41931f71f0824e293b7863063022de5a56dc27b2fde378c7b"} Sep 05 06:41:12.858954 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:12.858917 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:41:12.859238 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:12.858992 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:41:12.859238 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:12.859025 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:41:13.887928 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:13.887860 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" event={"ID":"59b55d7e-3fd4-41db-be04-452970ecab05","Type":"ContainerStarted","Data":"90f589771e2288b796f4cc9598f327d16384fb9c9aff8df5ecd73342721f3ef8"} Sep 05 06:41:13.921917 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:13.921866 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" podStartSLOduration=2.223565796 podStartE2EDuration="4.921850306s" totalImagesPullingTime="2.69828451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:41:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:41:13.9197104 +0000 UTC m=+970.603215323" watchObservedRunningTime="2026-09-05 06:41:13.921850306 +0000 UTC m=+970.605355214" Sep 05 06:41:14.871594 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:14.871553 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:14.876840 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:14.876813 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:14.893621 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:14.893582 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:14.894526 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:14.894471 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:15.669800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.669767 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2"] Sep 05 06:41:15.673890 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.673866 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.676956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.676928 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-ea7120c2-serving-cert\"" Sep 05 06:41:15.686445 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.686419 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2"] Sep 05 06:41:15.807886 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.807856 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-maas-api-tls\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.808067 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.807916 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-metrics-certs\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.808067 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.808008 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbx66\" (UniqueName: \"kubernetes.io/projected/794f0677-dce0-4de6-8dbc-a59fcc57daba-kube-api-access-vbx66\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.808185 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.808065 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/794f0677-dce0-4de6-8dbc-a59fcc57daba-openshift-service-ca\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.908889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.908853 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-maas-api-tls\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.909343 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.908907 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-metrics-certs\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.909343 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.908933 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbx66\" (UniqueName: \"kubernetes.io/projected/794f0677-dce0-4de6-8dbc-a59fcc57daba-kube-api-access-vbx66\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.909343 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.908969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/794f0677-dce0-4de6-8dbc-a59fcc57daba-openshift-service-ca\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.909343 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:41:15.909028 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-serving-cert: secret "maas-api-e2e-migrate-ea7120c2-serving-cert" not found Sep 05 06:41:15.909343 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:41:15.909118 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-maas-api-tls podName:794f0677-dce0-4de6-8dbc-a59fcc57daba nodeName:}" failed. No retries permitted until 2026-09-05 06:41:16.409096556 +0000 UTC m=+973.092601443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-maas-api-tls") pod "maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" (UID: "794f0677-dce0-4de6-8dbc-a59fcc57daba") : secret "maas-api-e2e-migrate-ea7120c2-serving-cert" not found Sep 05 06:41:15.909739 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.909719 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/794f0677-dce0-4de6-8dbc-a59fcc57daba-openshift-service-ca\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.911321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.911302 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-metrics-certs\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:15.918367 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:15.918343 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbx66\" (UniqueName: \"kubernetes.io/projected/794f0677-dce0-4de6-8dbc-a59fcc57daba-kube-api-access-vbx66\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:16.413791 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:16.413750 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-maas-api-tls\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:16.416135 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:16.416111 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-maas-api-tls\") pod \"maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:16.588812 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:16.588776 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:16.899104 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:16.899079 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2"] Sep 05 06:41:16.901650 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:41:16.901614 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794f0677_dce0_4de6_8dbc_a59fcc57daba.slice/crio-2a4b36628ddb5ebaabf6e1564d534fdc6ef308fd627f20c2375e89da1ce6652b WatchSource:0}: Error finding container 2a4b36628ddb5ebaabf6e1564d534fdc6ef308fd627f20c2375e89da1ce6652b: Status 404 returned error can't find the container with id 2a4b36628ddb5ebaabf6e1564d534fdc6ef308fd627f20c2375e89da1ce6652b Sep 05 06:41:17.914494 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:17.914441 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" event={"ID":"794f0677-dce0-4de6-8dbc-a59fcc57daba","Type":"ContainerStarted","Data":"664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee"} Sep 05 06:41:17.914970 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:17.914554 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:17.914970 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:17.914585 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" event={"ID":"794f0677-dce0-4de6-8dbc-a59fcc57daba","Type":"ContainerStarted","Data":"2a4b36628ddb5ebaabf6e1564d534fdc6ef308fd627f20c2375e89da1ce6652b"} Sep 05 06:41:17.942520 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:17.942433 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" podStartSLOduration=2.6301784379999997 podStartE2EDuration="2.942411399s" totalImagesPullingTime="312.232961ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:41:17.938963827 +0000 UTC m=+974.622468735" watchObservedRunningTime="2026-09-05 06:41:17.942411399 +0000 UTC m=+974.625916308" Sep 05 06:41:23.926970 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:23.926935 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:34.010880 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:34.010845 2583 generic.go:356] "Generic (PLEG): container finished" podID="59b55d7e-3fd4-41db-be04-452970ecab05" containerID="90f589771e2288b796f4cc9598f327d16384fb9c9aff8df5ecd73342721f3ef8" exitCode=0 Sep 05 06:41:34.011277 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:34.010915 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" event={"ID":"59b55d7e-3fd4-41db-be04-452970ecab05","Type":"ContainerDied","Data":"90f589771e2288b796f4cc9598f327d16384fb9c9aff8df5ecd73342721f3ef8"} Sep 05 06:41:34.011348 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:34.011333 2583 scope.go:117] "RemoveContainer" containerID="90f589771e2288b796f4cc9598f327d16384fb9c9aff8df5ecd73342721f3ef8" Sep 05 06:41:34.013321 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:34.013280 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:41:34.013445 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:34.013363 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:41:34.013445 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:34.013412 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:41:35.024156 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:35.024113 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" event={"ID":"59b55d7e-3fd4-41db-be04-452970ecab05","Type":"ContainerStarted","Data":"2f7880bec9b115b7664617760ee64424a14a303fe0b6c6f211709d58ab1daa70"} Sep 05 06:41:35.871991 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:35.871951 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:35.872280 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:35.872255 2583 patch_prober.go:28] interesting pod/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 05 06:41:35.872374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:35.872314 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 05 06:41:36.872721 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:36.872683 2583 patch_prober.go:28] interesting pod/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 05 06:41:36.873099 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:36.872747 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 05 06:41:37.872228 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:37.872193 2583 patch_prober.go:28] interesting pod/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 05 06:41:37.872400 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:37.872260 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 05 06:41:38.872419 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:38.872368 2583 patch_prober.go:28] interesting pod/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 05 06:41:38.873090 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:38.872470 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 05 06:41:39.872415 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:39.872380 2583 patch_prober.go:28] interesting pod/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 05 06:41:39.872609 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:39.872431 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 05 06:41:40.876503 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:40.876463 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:40.876958 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:40.876945 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:40.877673 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:40.877650 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:41:50.450564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.450523 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9"] Sep 05 06:41:50.451095 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.450804 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" podUID="d64cd300-a3e9-42ff-80f2-9ae248badb05" containerName="maas-api" containerID="cri-o://17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c" gracePeriod=30 Sep 05 06:41:50.712238 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.712179 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:41:50.766674 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.766642 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-maas-api-tls\") pod \"d64cd300-a3e9-42ff-80f2-9ae248badb05\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " Sep 05 06:41:50.766858 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.766690 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfsqx\" (UniqueName: \"kubernetes.io/projected/d64cd300-a3e9-42ff-80f2-9ae248badb05-kube-api-access-jfsqx\") pod \"d64cd300-a3e9-42ff-80f2-9ae248badb05\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " Sep 05 06:41:50.766858 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.766715 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-metrics-certs\") pod \"d64cd300-a3e9-42ff-80f2-9ae248badb05\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " Sep 05 06:41:50.766858 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.766793 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64cd300-a3e9-42ff-80f2-9ae248badb05-openshift-service-ca\") pod \"d64cd300-a3e9-42ff-80f2-9ae248badb05\" (UID: \"d64cd300-a3e9-42ff-80f2-9ae248badb05\") " Sep 05 06:41:50.767227 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.767191 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d64cd300-a3e9-42ff-80f2-9ae248badb05-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d64cd300-a3e9-42ff-80f2-9ae248badb05" (UID: "d64cd300-a3e9-42ff-80f2-9ae248badb05"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:41:50.768787 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.768759 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d64cd300-a3e9-42ff-80f2-9ae248badb05" (UID: "d64cd300-a3e9-42ff-80f2-9ae248badb05"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:41:50.768889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.768802 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d64cd300-a3e9-42ff-80f2-9ae248badb05" (UID: "d64cd300-a3e9-42ff-80f2-9ae248badb05"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:41:50.768889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.768829 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d64cd300-a3e9-42ff-80f2-9ae248badb05-kube-api-access-jfsqx" (OuterVolumeSpecName: "kube-api-access-jfsqx") pod "d64cd300-a3e9-42ff-80f2-9ae248badb05" (UID: "d64cd300-a3e9-42ff-80f2-9ae248badb05"). InnerVolumeSpecName "kube-api-access-jfsqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:41:50.868821 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.868723 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:50.868821 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.868761 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfsqx\" (UniqueName: \"kubernetes.io/projected/d64cd300-a3e9-42ff-80f2-9ae248badb05-kube-api-access-jfsqx\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:50.868821 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.868775 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d64cd300-a3e9-42ff-80f2-9ae248badb05-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:50.868821 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:50.868789 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d64cd300-a3e9-42ff-80f2-9ae248badb05-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:51.129248 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:51.129211 2583 generic.go:356] "Generic (PLEG): container finished" podID="d64cd300-a3e9-42ff-80f2-9ae248badb05" containerID="17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c" exitCode=0 Sep 05 06:41:51.129432 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:51.129264 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" Sep 05 06:41:51.129432 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:51.129292 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" event={"ID":"d64cd300-a3e9-42ff-80f2-9ae248badb05","Type":"ContainerDied","Data":"17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c"} Sep 05 06:41:51.129432 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:51.129345 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9" event={"ID":"d64cd300-a3e9-42ff-80f2-9ae248badb05","Type":"ContainerDied","Data":"b77bbbd80a0ca71d5d3885ad8ec5904376727362fe6cdcace1120e200f615423"} Sep 05 06:41:51.129432 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:51.129367 2583 scope.go:117] "RemoveContainer" containerID="17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c" Sep 05 06:41:51.139489 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:51.139456 2583 scope.go:117] "RemoveContainer" containerID="17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c" Sep 05 06:41:51.139786 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:41:51.139764 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c\": container with ID starting with 17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c not found: ID does not exist" containerID="17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c" Sep 05 06:41:51.139869 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:51.139800 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c"} err="failed to get container status \"17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c\": rpc error: code = NotFound desc = could not find container \"17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c\": container with ID starting with 17f404c093971555c85c7f455a7b46c219cfd27e2948fd156fa516e53433d19c not found: ID does not exist" Sep 05 06:41:51.158620 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:51.158594 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9"] Sep 05 06:41:51.161313 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:51.161286 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e04a57-9cb8b6f75-n6vz9"] Sep 05 06:41:51.822936 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:51.822904 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d64cd300-a3e9-42ff-80f2-9ae248badb05" path="/var/lib/kubelet/pods/d64cd300-a3e9-42ff-80f2-9ae248badb05/volumes" Sep 05 06:41:56.602255 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.602217 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2"] Sep 05 06:41:56.603058 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.603027 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" podUID="794f0677-dce0-4de6-8dbc-a59fcc57daba" containerName="maas-api" containerID="cri-o://664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee" gracePeriod=30 Sep 05 06:41:56.867969 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.867886 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:56.928379 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.928350 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbx66\" (UniqueName: \"kubernetes.io/projected/794f0677-dce0-4de6-8dbc-a59fcc57daba-kube-api-access-vbx66\") pod \"794f0677-dce0-4de6-8dbc-a59fcc57daba\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " Sep 05 06:41:56.928591 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.928542 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/794f0677-dce0-4de6-8dbc-a59fcc57daba-openshift-service-ca\") pod \"794f0677-dce0-4de6-8dbc-a59fcc57daba\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " Sep 05 06:41:56.928591 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.928586 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-metrics-certs\") pod \"794f0677-dce0-4de6-8dbc-a59fcc57daba\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " Sep 05 06:41:56.928726 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.928615 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-maas-api-tls\") pod \"794f0677-dce0-4de6-8dbc-a59fcc57daba\" (UID: \"794f0677-dce0-4de6-8dbc-a59fcc57daba\") " Sep 05 06:41:56.928959 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.928930 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/794f0677-dce0-4de6-8dbc-a59fcc57daba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "794f0677-dce0-4de6-8dbc-a59fcc57daba" (UID: "794f0677-dce0-4de6-8dbc-a59fcc57daba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:41:56.930678 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.930645 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/794f0677-dce0-4de6-8dbc-a59fcc57daba-kube-api-access-vbx66" (OuterVolumeSpecName: "kube-api-access-vbx66") pod "794f0677-dce0-4de6-8dbc-a59fcc57daba" (UID: "794f0677-dce0-4de6-8dbc-a59fcc57daba"). InnerVolumeSpecName "kube-api-access-vbx66". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:41:56.930678 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.930665 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "794f0677-dce0-4de6-8dbc-a59fcc57daba" (UID: "794f0677-dce0-4de6-8dbc-a59fcc57daba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:41:56.930818 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:56.930716 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "794f0677-dce0-4de6-8dbc-a59fcc57daba" (UID: "794f0677-dce0-4de6-8dbc-a59fcc57daba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:41:57.029795 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.029758 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/794f0677-dce0-4de6-8dbc-a59fcc57daba-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:57.029795 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.029795 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:57.030008 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.029809 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/794f0677-dce0-4de6-8dbc-a59fcc57daba-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:57.030008 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.029820 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbx66\" (UniqueName: \"kubernetes.io/projected/794f0677-dce0-4de6-8dbc-a59fcc57daba-kube-api-access-vbx66\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:41:57.042218 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.042181 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz"] Sep 05 06:41:57.042499 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.042436 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" containerID="cri-o://2f7880bec9b115b7664617760ee64424a14a303fe0b6c6f211709d58ab1daa70" gracePeriod=30 Sep 05 06:41:57.169547 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.169370 2583 generic.go:356] "Generic (PLEG): container finished" podID="794f0677-dce0-4de6-8dbc-a59fcc57daba" containerID="664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee" exitCode=0 Sep 05 06:41:57.169547 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.169410 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" Sep 05 06:41:57.169547 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.169414 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" event={"ID":"794f0677-dce0-4de6-8dbc-a59fcc57daba","Type":"ContainerDied","Data":"664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee"} Sep 05 06:41:57.169547 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.169495 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2" event={"ID":"794f0677-dce0-4de6-8dbc-a59fcc57daba","Type":"ContainerDied","Data":"2a4b36628ddb5ebaabf6e1564d534fdc6ef308fd627f20c2375e89da1ce6652b"} Sep 05 06:41:57.169547 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.169529 2583 scope.go:117] "RemoveContainer" containerID="664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee" Sep 05 06:41:57.180000 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.179983 2583 scope.go:117] "RemoveContainer" containerID="664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee" Sep 05 06:41:57.180265 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:41:57.180244 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee\": container with ID starting with 664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee not found: ID does not exist" containerID="664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee" Sep 05 06:41:57.180313 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.180275 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee"} err="failed to get container status \"664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee\": rpc error: code = NotFound desc = could not find container \"664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee\": container with ID starting with 664a75786d559c6a201648a7859e3e3eccf2fec5ca81a8a1deaa0e404adf45ee not found: ID does not exist" Sep 05 06:41:57.194295 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.194270 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2"] Sep 05 06:41:57.198543 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.198523 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ea7120c2-7b7956c7b6-8mxw2"] Sep 05 06:41:57.824539 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:57.824469 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="794f0677-dce0-4de6-8dbc-a59fcc57daba" path="/var/lib/kubelet/pods/794f0677-dce0-4de6-8dbc-a59fcc57daba/volumes" Sep 05 06:41:59.301391 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.301358 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw"] Sep 05 06:41:59.301850 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.301836 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794f0677-dce0-4de6-8dbc-a59fcc57daba" containerName="maas-api" Sep 05 06:41:59.301893 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.301853 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="794f0677-dce0-4de6-8dbc-a59fcc57daba" containerName="maas-api" Sep 05 06:41:59.301893 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.301868 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d64cd300-a3e9-42ff-80f2-9ae248badb05" containerName="maas-api" Sep 05 06:41:59.301893 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.301874 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d64cd300-a3e9-42ff-80f2-9ae248badb05" containerName="maas-api" Sep 05 06:41:59.301987 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.301951 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d64cd300-a3e9-42ff-80f2-9ae248badb05" containerName="maas-api" Sep 05 06:41:59.301987 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.301963 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="794f0677-dce0-4de6-8dbc-a59fcc57daba" containerName="maas-api" Sep 05 06:41:59.306544 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.306524 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.309573 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.309544 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-a092eb-openshift-default-dockercfg-ssqsc\"" Sep 05 06:41:59.325972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.325948 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw"] Sep 05 06:41:59.454782 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.454739 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-token\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.454975 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.454799 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istiod-ca-cert\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.454975 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.454877 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-certs\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.454975 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.454965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-credential-socket\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.455101 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.455014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-envoy\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.455101 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.455035 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-socket\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.455178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.455123 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-data\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.455178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.455141 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bblt4\" (UniqueName: \"kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-kube-api-access-bblt4\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.455178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.455167 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-podinfo\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.556609 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.556526 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istiod-ca-cert\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.556609 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.556567 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-certs\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.556824 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.556692 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-credential-socket\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.556824 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.556748 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-envoy\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.556824 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.556771 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-socket\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.556983 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.556884 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-data\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.556983 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.556918 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bblt4\" (UniqueName: \"kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-kube-api-access-bblt4\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.556983 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.556954 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-podinfo\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.557122 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.556984 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-token\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.557122 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.557070 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-certs\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.557366 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.557305 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-data\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.557366 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.557340 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istiod-ca-cert\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.557550 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.557384 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-socket\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.557550 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.557400 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-credential-socket\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.559280 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.559253 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-envoy\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.559488 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.559458 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-podinfo\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.566561 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.566540 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-token\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.567333 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.567313 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bblt4\" (UniqueName: \"kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-kube-api-access-bblt4\") pod \"e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.617045 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.617022 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:41:59.897949 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.897924 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw"] Sep 05 06:41:59.898691 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:41:59.898663 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3bc3eae_0d6d_47a8_989d_b45774dc3664.slice/crio-917efbf4ea77c7cf760761b01117323bb2046eb4196c2109bcf056a67687827c WatchSource:0}: Error finding container 917efbf4ea77c7cf760761b01117323bb2046eb4196c2109bcf056a67687827c: Status 404 returned error can't find the container with id 917efbf4ea77c7cf760761b01117323bb2046eb4196c2109bcf056a67687827c Sep 05 06:41:59.900989 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.900954 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:41:59.901092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.901015 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:41:59.901092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:41:59.901044 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:42:00.191504 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:00.191370 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" event={"ID":"c3bc3eae-0d6d-47a8-989d-b45774dc3664","Type":"ContainerStarted","Data":"743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5"} Sep 05 06:42:00.191504 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:00.191437 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" event={"ID":"c3bc3eae-0d6d-47a8-989d-b45774dc3664","Type":"ContainerStarted","Data":"917efbf4ea77c7cf760761b01117323bb2046eb4196c2109bcf056a67687827c"} Sep 05 06:42:00.215376 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:00.215311 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" podStartSLOduration=1.215289703 podStartE2EDuration="1.215289703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:41:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:42:00.214503638 +0000 UTC m=+1016.898008564" watchObservedRunningTime="2026-09-05 06:42:00.215289703 +0000 UTC m=+1016.898794611" Sep 05 06:42:00.617638 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:00.617599 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:42:00.623564 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:00.623537 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:42:01.197398 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:01.197361 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:42:01.198446 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:01.198428 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:42:01.848695 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:01.848665 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk"] Sep 05 06:42:01.856120 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:01.856018 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:01.862659 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:01.862631 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-46fa710a-serving-cert\"" Sep 05 06:42:01.868963 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:01.868932 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk"] Sep 05 06:42:01.985966 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:01.985934 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/697146a3-f38f-4839-868b-9c59b2a29fe2-openshift-service-ca\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:01.986118 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:01.986025 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-metrics-certs\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:01.986118 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:01.986053 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l27bj\" (UniqueName: \"kubernetes.io/projected/697146a3-f38f-4839-868b-9c59b2a29fe2-kube-api-access-l27bj\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:01.986118 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:01.986092 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-maas-api-tls\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:02.087599 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.087556 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/697146a3-f38f-4839-868b-9c59b2a29fe2-openshift-service-ca\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:02.087741 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.087654 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-metrics-certs\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:02.087741 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.087676 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l27bj\" (UniqueName: \"kubernetes.io/projected/697146a3-f38f-4839-868b-9c59b2a29fe2-kube-api-access-l27bj\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:02.087741 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.087709 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-maas-api-tls\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:02.088308 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.088282 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/697146a3-f38f-4839-868b-9c59b2a29fe2-openshift-service-ca\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:02.090261 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.090239 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-maas-api-tls\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:02.090330 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.090239 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-metrics-certs\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:02.099427 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.099378 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l27bj\" (UniqueName: \"kubernetes.io/projected/697146a3-f38f-4839-868b-9c59b2a29fe2-kube-api-access-l27bj\") pod \"maas-api-e2e-ait-46fa710a-58f85c77b-s6srk\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:02.176010 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.175976 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:02.210199 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.210171 2583 generic.go:356] "Generic (PLEG): container finished" podID="59b55d7e-3fd4-41db-be04-452970ecab05" containerID="2f7880bec9b115b7664617760ee64424a14a303fe0b6c6f211709d58ab1daa70" exitCode=0 Sep 05 06:42:02.210750 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.210290 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" event={"ID":"59b55d7e-3fd4-41db-be04-452970ecab05","Type":"ContainerDied","Data":"2f7880bec9b115b7664617760ee64424a14a303fe0b6c6f211709d58ab1daa70"} Sep 05 06:42:02.210858 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.210810 2583 scope.go:117] "RemoveContainer" containerID="90f589771e2288b796f4cc9598f327d16384fb9c9aff8df5ecd73342721f3ef8" Sep 05 06:42:02.324738 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.324715 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:42:02.473627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.473600 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk"] Sep 05 06:42:02.474932 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:42:02.474905 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod697146a3_f38f_4839_868b_9c59b2a29fe2.slice/crio-4789bb0ac83e7209a7a4264c79dbbce27707c537485fa432971b3fc301a3e27a WatchSource:0}: Error finding container 4789bb0ac83e7209a7a4264c79dbbce27707c537485fa432971b3fc301a3e27a: Status 404 returned error can't find the container with id 4789bb0ac83e7209a7a4264c79dbbce27707c537485fa432971b3fc301a3e27a Sep 05 06:42:02.490923 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.490902 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpbgn\" (UniqueName: \"kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-kube-api-access-bpbgn\") pod \"59b55d7e-3fd4-41db-be04-452970ecab05\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " Sep 05 06:42:02.491026 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.490968 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-envoy\") pod \"59b55d7e-3fd4-41db-be04-452970ecab05\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " Sep 05 06:42:02.491137 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491114 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59b55d7e-3fd4-41db-be04-452970ecab05-istiod-ca-cert\") pod \"59b55d7e-3fd4-41db-be04-452970ecab05\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " Sep 05 06:42:02.491219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491160 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-certs\") pod \"59b55d7e-3fd4-41db-be04-452970ecab05\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " Sep 05 06:42:02.491219 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491203 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-data\") pod \"59b55d7e-3fd4-41db-be04-452970ecab05\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " Sep 05 06:42:02.491325 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491251 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59b55d7e-3fd4-41db-be04-452970ecab05-istio-podinfo\") pod \"59b55d7e-3fd4-41db-be04-452970ecab05\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " Sep 05 06:42:02.491325 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491285 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-socket\") pod \"59b55d7e-3fd4-41db-be04-452970ecab05\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " Sep 05 06:42:02.491441 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491341 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-credential-socket\") pod \"59b55d7e-3fd4-41db-be04-452970ecab05\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " Sep 05 06:42:02.491441 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491367 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-istio-token\") pod \"59b55d7e-3fd4-41db-be04-452970ecab05\" (UID: \"59b55d7e-3fd4-41db-be04-452970ecab05\") " Sep 05 06:42:02.491605 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491429 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59b55d7e-3fd4-41db-be04-452970ecab05" (UID: "59b55d7e-3fd4-41db-be04-452970ecab05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:42:02.491605 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491465 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-data" (OuterVolumeSpecName: "istio-data") pod "59b55d7e-3fd4-41db-be04-452970ecab05" (UID: "59b55d7e-3fd4-41db-be04-452970ecab05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:42:02.491714 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491645 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59b55d7e-3fd4-41db-be04-452970ecab05" (UID: "59b55d7e-3fd4-41db-be04-452970ecab05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:42:02.491714 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491664 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59b55d7e-3fd4-41db-be04-452970ecab05" (UID: "59b55d7e-3fd4-41db-be04-452970ecab05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:42:02.491838 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491725 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59b55d7e-3fd4-41db-be04-452970ecab05-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59b55d7e-3fd4-41db-be04-452970ecab05" (UID: "59b55d7e-3fd4-41db-be04-452970ecab05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:42:02.491894 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491835 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:02.491894 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491850 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-data\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:02.491894 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491868 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-workload-socket\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:02.491894 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.491882 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-credential-socket\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:02.493696 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.493649 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59b55d7e-3fd4-41db-be04-452970ecab05-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59b55d7e-3fd4-41db-be04-452970ecab05" (UID: "59b55d7e-3fd4-41db-be04-452970ecab05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 05 06:42:02.493800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.493689 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-istio-token" (OuterVolumeSpecName: "istio-token") pod "59b55d7e-3fd4-41db-be04-452970ecab05" (UID: "59b55d7e-3fd4-41db-be04-452970ecab05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:42:02.493800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.493764 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59b55d7e-3fd4-41db-be04-452970ecab05" (UID: "59b55d7e-3fd4-41db-be04-452970ecab05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:42:02.493800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.493762 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-kube-api-access-bpbgn" (OuterVolumeSpecName: "kube-api-access-bpbgn") pod "59b55d7e-3fd4-41db-be04-452970ecab05" (UID: "59b55d7e-3fd4-41db-be04-452970ecab05"). InnerVolumeSpecName "kube-api-access-bpbgn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:42:02.592966 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.592923 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpbgn\" (UniqueName: \"kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-kube-api-access-bpbgn\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:02.592966 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.592956 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59b55d7e-3fd4-41db-be04-452970ecab05-istio-envoy\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:02.592966 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.592970 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59b55d7e-3fd4-41db-be04-452970ecab05-istiod-ca-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:02.593199 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.592981 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59b55d7e-3fd4-41db-be04-452970ecab05-istio-podinfo\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:02.593199 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:02.592993 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59b55d7e-3fd4-41db-be04-452970ecab05-istio-token\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:03.218994 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.218916 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" Sep 05 06:42:03.219440 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.218915 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz" event={"ID":"59b55d7e-3fd4-41db-be04-452970ecab05","Type":"ContainerDied","Data":"beb122f290e621e41931f71f0824e293b7863063022de5a56dc27b2fde378c7b"} Sep 05 06:42:03.219440 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.219062 2583 scope.go:117] "RemoveContainer" containerID="2f7880bec9b115b7664617760ee64424a14a303fe0b6c6f211709d58ab1daa70" Sep 05 06:42:03.220992 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.220965 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" event={"ID":"697146a3-f38f-4839-868b-9c59b2a29fe2","Type":"ContainerStarted","Data":"26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e"} Sep 05 06:42:03.221193 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.221175 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:03.221271 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.221221 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" event={"ID":"697146a3-f38f-4839-868b-9c59b2a29fe2","Type":"ContainerStarted","Data":"4789bb0ac83e7209a7a4264c79dbbce27707c537485fa432971b3fc301a3e27a"} Sep 05 06:42:03.253854 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.253807 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" podStartSLOduration=1.858853374 podStartE2EDuration="2.253793651s" totalImagesPullingTime="394.940277ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:42:03.250083882 +0000 UTC m=+1019.933588792" watchObservedRunningTime="2026-09-05 06:42:03.253793651 +0000 UTC m=+1019.937298557" Sep 05 06:42:03.270551 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.270525 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz"] Sep 05 06:42:03.277202 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.277181 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ea7120c2-gw-openshift-default-58b6ff5668-t8qhz"] Sep 05 06:42:03.664174 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.664140 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g"] Sep 05 06:42:03.664829 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.664812 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" Sep 05 06:42:03.664904 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.664831 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" Sep 05 06:42:03.664976 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.664965 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" Sep 05 06:42:03.665016 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.664982 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" Sep 05 06:42:03.669909 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.669889 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.674462 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.674442 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-a092eb-serving-cert\"" Sep 05 06:42:03.685855 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.685830 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g"] Sep 05 06:42:03.804413 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.804376 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9deef366-9fca-4cde-a2a4-21bf34e05efa-openshift-service-ca\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.804635 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.804458 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg9vx\" (UniqueName: \"kubernetes.io/projected/9deef366-9fca-4cde-a2a4-21bf34e05efa-kube-api-access-mg9vx\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.804635 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.804507 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-maas-api-tls\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.804635 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.804535 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-metrics-certs\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.827063 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.827028 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" path="/var/lib/kubelet/pods/59b55d7e-3fd4-41db-be04-452970ecab05/volumes" Sep 05 06:42:03.905431 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.905403 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9deef366-9fca-4cde-a2a4-21bf34e05efa-openshift-service-ca\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.905595 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.905579 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg9vx\" (UniqueName: \"kubernetes.io/projected/9deef366-9fca-4cde-a2a4-21bf34e05efa-kube-api-access-mg9vx\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.905659 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.905647 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-maas-api-tls\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.905991 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.905810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-metrics-certs\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.906147 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.906130 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9deef366-9fca-4cde-a2a4-21bf34e05efa-openshift-service-ca\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.908131 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.908111 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-metrics-certs\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.908640 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.908624 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-a092eb-serving-cert\"" Sep 05 06:42:03.916631 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.916565 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg9vx\" (UniqueName: \"kubernetes.io/projected/9deef366-9fca-4cde-a2a4-21bf34e05efa-kube-api-access-mg9vx\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.918343 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.918319 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-maas-api-tls\") pod \"maas-api-e2e-explicit-a092eb-757f76455-2m96g\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:03.981725 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:03.981705 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:04.274936 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:04.274903 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g"] Sep 05 06:42:04.276083 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:42:04.276057 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9deef366_9fca_4cde_a2a4_21bf34e05efa.slice/crio-b1d8388438ed039fe66c6d24030e3431d26eb4ad476fd843929d913535c79396 WatchSource:0}: Error finding container b1d8388438ed039fe66c6d24030e3431d26eb4ad476fd843929d913535c79396: Status 404 returned error can't find the container with id b1d8388438ed039fe66c6d24030e3431d26eb4ad476fd843929d913535c79396 Sep 05 06:42:05.234672 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:05.234634 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" event={"ID":"9deef366-9fca-4cde-a2a4-21bf34e05efa","Type":"ContainerStarted","Data":"e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8"} Sep 05 06:42:05.234873 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:05.234734 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" event={"ID":"9deef366-9fca-4cde-a2a4-21bf34e05efa","Type":"ContainerStarted","Data":"b1d8388438ed039fe66c6d24030e3431d26eb4ad476fd843929d913535c79396"} Sep 05 06:42:05.234873 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:05.234763 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:05.256139 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:05.256086 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" podStartSLOduration=1.878449673 podStartE2EDuration="2.256070845s" totalImagesPullingTime="377.621172ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:42:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:42:05.253272089 +0000 UTC m=+1021.936776996" watchObservedRunningTime="2026-09-05 06:42:05.256070845 +0000 UTC m=+1021.939575751" Sep 05 06:42:09.233023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:09.232990 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:11.245965 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:11.245935 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:42:27.375969 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:27.375937 2583 generic.go:356] "Generic (PLEG): container finished" podID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerID="743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5" exitCode=0 Sep 05 06:42:27.376362 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:27.375976 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" event={"ID":"c3bc3eae-0d6d-47a8-989d-b45774dc3664","Type":"ContainerDied","Data":"743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5"} Sep 05 06:42:27.376427 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:27.376407 2583 scope.go:117] "RemoveContainer" containerID="743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5" Sep 05 06:42:27.377525 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:27.377468 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:42:27.377643 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:27.377580 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:42:27.377643 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:27.377632 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:42:28.383807 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:28.383771 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" event={"ID":"c3bc3eae-0d6d-47a8-989d-b45774dc3664","Type":"ContainerStarted","Data":"64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d"} Sep 05 06:42:28.618062 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:28.618025 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:42:28.619367 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:28.619335 2583 patch_prober.go:28] interesting pod/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Sep 05 06:42:28.619611 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:28.619573 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Sep 05 06:42:29.618228 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:29.618186 2583 patch_prober.go:28] interesting pod/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Sep 05 06:42:29.618859 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:29.618824 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Sep 05 06:42:30.618161 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:30.618117 2583 patch_prober.go:28] interesting pod/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Sep 05 06:42:30.618388 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:30.618188 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Sep 05 06:42:31.197601 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:31.197568 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:42:31.618526 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:31.618467 2583 patch_prober.go:28] interesting pod/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Sep 05 06:42:31.618990 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:31.618557 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Sep 05 06:42:32.618289 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:32.617848 2583 patch_prober.go:28] interesting pod/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Sep 05 06:42:32.618289 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:32.617913 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Sep 05 06:42:32.695935 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:32.695902 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk"] Sep 05 06:42:32.696386 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:32.696113 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" podUID="697146a3-f38f-4839-868b-9c59b2a29fe2" containerName="maas-api" containerID="cri-o://26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e" gracePeriod=30 Sep 05 06:42:32.999674 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:32.999650 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:33.089302 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.089269 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/697146a3-f38f-4839-868b-9c59b2a29fe2-openshift-service-ca\") pod \"697146a3-f38f-4839-868b-9c59b2a29fe2\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " Sep 05 06:42:33.089515 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.089326 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-maas-api-tls\") pod \"697146a3-f38f-4839-868b-9c59b2a29fe2\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " Sep 05 06:42:33.089515 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.089386 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-metrics-certs\") pod \"697146a3-f38f-4839-868b-9c59b2a29fe2\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " Sep 05 06:42:33.089515 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.089418 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l27bj\" (UniqueName: \"kubernetes.io/projected/697146a3-f38f-4839-868b-9c59b2a29fe2-kube-api-access-l27bj\") pod \"697146a3-f38f-4839-868b-9c59b2a29fe2\" (UID: \"697146a3-f38f-4839-868b-9c59b2a29fe2\") " Sep 05 06:42:33.089655 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.089614 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/697146a3-f38f-4839-868b-9c59b2a29fe2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "697146a3-f38f-4839-868b-9c59b2a29fe2" (UID: "697146a3-f38f-4839-868b-9c59b2a29fe2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:42:33.091671 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.091647 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/697146a3-f38f-4839-868b-9c59b2a29fe2-kube-api-access-l27bj" (OuterVolumeSpecName: "kube-api-access-l27bj") pod "697146a3-f38f-4839-868b-9c59b2a29fe2" (UID: "697146a3-f38f-4839-868b-9c59b2a29fe2"). InnerVolumeSpecName "kube-api-access-l27bj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:42:33.091782 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.091698 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "697146a3-f38f-4839-868b-9c59b2a29fe2" (UID: "697146a3-f38f-4839-868b-9c59b2a29fe2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:42:33.092107 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.092086 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "697146a3-f38f-4839-868b-9c59b2a29fe2" (UID: "697146a3-f38f-4839-868b-9c59b2a29fe2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:42:33.190879 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.190804 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/697146a3-f38f-4839-868b-9c59b2a29fe2-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:33.190879 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.190830 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:33.190879 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.190839 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/697146a3-f38f-4839-868b-9c59b2a29fe2-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:33.190879 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.190848 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l27bj\" (UniqueName: \"kubernetes.io/projected/697146a3-f38f-4839-868b-9c59b2a29fe2-kube-api-access-l27bj\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:42:33.422292 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.422249 2583 generic.go:356] "Generic (PLEG): container finished" podID="697146a3-f38f-4839-868b-9c59b2a29fe2" containerID="26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e" exitCode=0 Sep 05 06:42:33.422292 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.422300 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" Sep 05 06:42:33.422574 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.422447 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" event={"ID":"697146a3-f38f-4839-868b-9c59b2a29fe2","Type":"ContainerDied","Data":"26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e"} Sep 05 06:42:33.422574 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.422512 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk" event={"ID":"697146a3-f38f-4839-868b-9c59b2a29fe2","Type":"ContainerDied","Data":"4789bb0ac83e7209a7a4264c79dbbce27707c537485fa432971b3fc301a3e27a"} Sep 05 06:42:33.422574 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.422544 2583 scope.go:117] "RemoveContainer" containerID="26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e" Sep 05 06:42:33.437267 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.437249 2583 scope.go:117] "RemoveContainer" containerID="26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e" Sep 05 06:42:33.437557 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:42:33.437507 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e\": container with ID starting with 26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e not found: ID does not exist" containerID="26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e" Sep 05 06:42:33.437557 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.437536 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e"} err="failed to get container status \"26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e\": rpc error: code = NotFound desc = could not find container \"26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e\": container with ID starting with 26e5420457c2a38a1f9dfd5dcb999713d74c3cc04ff8c1d45c998afda3a48c9e not found: ID does not exist" Sep 05 06:42:33.449562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.449469 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk"] Sep 05 06:42:33.456279 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.456216 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-46fa710a-58f85c77b-s6srk"] Sep 05 06:42:33.618171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.618135 2583 patch_prober.go:28] interesting pod/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Sep 05 06:42:33.618850 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.618438 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Sep 05 06:42:33.823561 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:33.823529 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="697146a3-f38f-4839-868b-9c59b2a29fe2" path="/var/lib/kubelet/pods/697146a3-f38f-4839-868b-9c59b2a29fe2/volumes" Sep 05 06:42:34.622084 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:34.622047 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:42:34.623076 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:34.623054 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:42:38.980778 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:38.980745 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx"] Sep 05 06:42:38.981240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:38.981225 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="697146a3-f38f-4839-868b-9c59b2a29fe2" containerName="maas-api" Sep 05 06:42:38.981306 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:38.981243 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="697146a3-f38f-4839-868b-9c59b2a29fe2" containerName="maas-api" Sep 05 06:42:38.981306 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:38.981257 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" Sep 05 06:42:38.981306 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:38.981263 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="59b55d7e-3fd4-41db-be04-452970ecab05" containerName="istio-proxy" Sep 05 06:42:38.981458 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:38.981365 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="697146a3-f38f-4839-868b-9c59b2a29fe2" containerName="maas-api" Sep 05 06:42:38.986175 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:38.986150 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:38.989124 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:38.989101 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a40054de-serving-cert\"" Sep 05 06:42:38.996968 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:38.996945 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx"] Sep 05 06:42:39.045917 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.045884 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-maas-api-tls\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.046059 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.045922 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjtj5\" (UniqueName: \"kubernetes.io/projected/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-kube-api-access-sjtj5\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.046059 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.046043 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-metrics-certs\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.046151 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.046087 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-openshift-service-ca\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.147623 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.147589 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-maas-api-tls\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.147776 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.147635 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjtj5\" (UniqueName: \"kubernetes.io/projected/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-kube-api-access-sjtj5\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.147776 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.147700 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-metrics-certs\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.147776 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.147730 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-openshift-service-ca\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.147776 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:42:39.147741 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-serving-cert: secret "maas-api-e2e-derive-a40054de-serving-cert" not found Sep 05 06:42:39.147942 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:42:39.147841 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-maas-api-tls podName:fbc04aee-bac0-47ec-bf0a-2dd40ce54859 nodeName:}" failed. No retries permitted until 2026-09-05 06:42:39.647818082 +0000 UTC m=+1056.331322980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-maas-api-tls") pod "maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" (UID: "fbc04aee-bac0-47ec-bf0a-2dd40ce54859") : secret "maas-api-e2e-derive-a40054de-serving-cert" not found Sep 05 06:42:39.148523 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.148501 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-openshift-service-ca\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.150255 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.150236 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-metrics-certs\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.160383 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.160360 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjtj5\" (UniqueName: \"kubernetes.io/projected/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-kube-api-access-sjtj5\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.653875 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.653827 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-maas-api-tls\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.656471 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.656441 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-maas-api-tls\") pod \"maas-api-e2e-derive-a40054de-756d6ff678-6gnkx\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:39.900243 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:39.900212 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:40.176206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:40.176176 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx"] Sep 05 06:42:40.177103 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:42:40.177078 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc04aee_bac0_47ec_bf0a_2dd40ce54859.slice/crio-cb6c08762d7389e1c45859829bc1bb994bd2fd0212016d047b4c00087998af42 WatchSource:0}: Error finding container cb6c08762d7389e1c45859829bc1bb994bd2fd0212016d047b4c00087998af42: Status 404 returned error can't find the container with id cb6c08762d7389e1c45859829bc1bb994bd2fd0212016d047b4c00087998af42 Sep 05 06:42:40.178909 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:40.178890 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 05 06:42:40.468201 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:40.468116 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" event={"ID":"fbc04aee-bac0-47ec-bf0a-2dd40ce54859","Type":"ContainerStarted","Data":"cb6c08762d7389e1c45859829bc1bb994bd2fd0212016d047b4c00087998af42"} Sep 05 06:42:41.475093 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:41.475056 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" event={"ID":"fbc04aee-bac0-47ec-bf0a-2dd40ce54859","Type":"ContainerStarted","Data":"027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67"} Sep 05 06:42:41.475562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:41.475118 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:42:41.494519 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:41.494458 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" podStartSLOduration=3.042141386 podStartE2EDuration="3.49444493s" totalImagesPullingTime="452.303544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:42:41.491960741 +0000 UTC m=+1058.175465648" watchObservedRunningTime="2026-09-05 06:42:41.49444493 +0000 UTC m=+1058.177949837" Sep 05 06:42:47.486296 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:42:47.486261 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:43:03.897468 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:03.897397 2583 scope.go:117] "RemoveContainer" containerID="1748cc97afa21dcaf7d5fc75bb6c6ae1252197580267de9dc0405cc141469036" Sep 05 06:43:09.790026 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:09.789988 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx"] Sep 05 06:43:09.790683 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:09.790338 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" podUID="fbc04aee-bac0-47ec-bf0a-2dd40ce54859" containerName="maas-api" containerID="cri-o://027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67" gracePeriod=30 Sep 05 06:43:10.050648 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.050588 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:43:10.146828 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.146793 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-openshift-service-ca\") pod \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " Sep 05 06:43:10.147001 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.146878 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-metrics-certs\") pod \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " Sep 05 06:43:10.147001 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.146954 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-maas-api-tls\") pod \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " Sep 05 06:43:10.147122 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.147077 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjtj5\" (UniqueName: \"kubernetes.io/projected/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-kube-api-access-sjtj5\") pod \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\" (UID: \"fbc04aee-bac0-47ec-bf0a-2dd40ce54859\") " Sep 05 06:43:10.147213 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.147187 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fbc04aee-bac0-47ec-bf0a-2dd40ce54859" (UID: "fbc04aee-bac0-47ec-bf0a-2dd40ce54859"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:43:10.147570 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.147543 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:10.149044 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.149012 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fbc04aee-bac0-47ec-bf0a-2dd40ce54859" (UID: "fbc04aee-bac0-47ec-bf0a-2dd40ce54859"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:43:10.149397 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.149370 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fbc04aee-bac0-47ec-bf0a-2dd40ce54859" (UID: "fbc04aee-bac0-47ec-bf0a-2dd40ce54859"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:43:10.149594 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.149567 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-kube-api-access-sjtj5" (OuterVolumeSpecName: "kube-api-access-sjtj5") pod "fbc04aee-bac0-47ec-bf0a-2dd40ce54859" (UID: "fbc04aee-bac0-47ec-bf0a-2dd40ce54859"). InnerVolumeSpecName "kube-api-access-sjtj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:43:10.248071 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.248038 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:10.248071 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.248066 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjtj5\" (UniqueName: \"kubernetes.io/projected/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-kube-api-access-sjtj5\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:10.248071 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.248076 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbc04aee-bac0-47ec-bf0a-2dd40ce54859-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:10.647943 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.647907 2583 generic.go:356] "Generic (PLEG): container finished" podID="fbc04aee-bac0-47ec-bf0a-2dd40ce54859" containerID="027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67" exitCode=0 Sep 05 06:43:10.648106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.647958 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" Sep 05 06:43:10.648106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.647965 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" event={"ID":"fbc04aee-bac0-47ec-bf0a-2dd40ce54859","Type":"ContainerDied","Data":"027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67"} Sep 05 06:43:10.648106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.648022 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx" event={"ID":"fbc04aee-bac0-47ec-bf0a-2dd40ce54859","Type":"ContainerDied","Data":"cb6c08762d7389e1c45859829bc1bb994bd2fd0212016d047b4c00087998af42"} Sep 05 06:43:10.648106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.648051 2583 scope.go:117] "RemoveContainer" containerID="027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67" Sep 05 06:43:10.658269 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.658249 2583 scope.go:117] "RemoveContainer" containerID="027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67" Sep 05 06:43:10.658576 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:43:10.658554 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67\": container with ID starting with 027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67 not found: ID does not exist" containerID="027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67" Sep 05 06:43:10.658652 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.658583 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67"} err="failed to get container status \"027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67\": rpc error: code = NotFound desc = could not find container \"027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67\": container with ID starting with 027dbe63671727161680c8786edcccce2b9e21ef1a7f4923d2eaf183c7cd6f67 not found: ID does not exist" Sep 05 06:43:10.672374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.672351 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx"] Sep 05 06:43:10.677056 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:10.677036 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a40054de-756d6ff678-6gnkx"] Sep 05 06:43:11.823252 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:11.823219 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbc04aee-bac0-47ec-bf0a-2dd40ce54859" path="/var/lib/kubelet/pods/fbc04aee-bac0-47ec-bf0a-2dd40ce54859/volumes" Sep 05 06:43:17.837313 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:17.837277 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g"] Sep 05 06:43:17.837713 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:17.837496 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" podUID="9deef366-9fca-4cde-a2a4-21bf34e05efa" containerName="maas-api" containerID="cri-o://e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8" gracePeriod=30 Sep 05 06:43:18.091898 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.091841 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:43:18.219682 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.219642 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-maas-api-tls\") pod \"9deef366-9fca-4cde-a2a4-21bf34e05efa\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " Sep 05 06:43:18.219859 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.219715 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg9vx\" (UniqueName: \"kubernetes.io/projected/9deef366-9fca-4cde-a2a4-21bf34e05efa-kube-api-access-mg9vx\") pod \"9deef366-9fca-4cde-a2a4-21bf34e05efa\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " Sep 05 06:43:18.219859 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.219805 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9deef366-9fca-4cde-a2a4-21bf34e05efa-openshift-service-ca\") pod \"9deef366-9fca-4cde-a2a4-21bf34e05efa\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " Sep 05 06:43:18.219985 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.219875 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-metrics-certs\") pod \"9deef366-9fca-4cde-a2a4-21bf34e05efa\" (UID: \"9deef366-9fca-4cde-a2a4-21bf34e05efa\") " Sep 05 06:43:18.222585 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.222512 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9deef366-9fca-4cde-a2a4-21bf34e05efa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9deef366-9fca-4cde-a2a4-21bf34e05efa" (UID: "9deef366-9fca-4cde-a2a4-21bf34e05efa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:43:18.225504 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.224099 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9deef366-9fca-4cde-a2a4-21bf34e05efa-kube-api-access-mg9vx" (OuterVolumeSpecName: "kube-api-access-mg9vx") pod "9deef366-9fca-4cde-a2a4-21bf34e05efa" (UID: "9deef366-9fca-4cde-a2a4-21bf34e05efa"). InnerVolumeSpecName "kube-api-access-mg9vx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:43:18.225504 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.224148 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9deef366-9fca-4cde-a2a4-21bf34e05efa" (UID: "9deef366-9fca-4cde-a2a4-21bf34e05efa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:43:18.225689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.225553 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9deef366-9fca-4cde-a2a4-21bf34e05efa" (UID: "9deef366-9fca-4cde-a2a4-21bf34e05efa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:43:18.321570 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.321522 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9deef366-9fca-4cde-a2a4-21bf34e05efa-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:18.321570 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.321564 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:18.321570 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.321576 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9deef366-9fca-4cde-a2a4-21bf34e05efa-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:18.321570 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.321585 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg9vx\" (UniqueName: \"kubernetes.io/projected/9deef366-9fca-4cde-a2a4-21bf34e05efa-kube-api-access-mg9vx\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:18.694905 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.694873 2583 generic.go:356] "Generic (PLEG): container finished" podID="9deef366-9fca-4cde-a2a4-21bf34e05efa" containerID="e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8" exitCode=0 Sep 05 06:43:18.695106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.694915 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" Sep 05 06:43:18.695106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.694959 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" event={"ID":"9deef366-9fca-4cde-a2a4-21bf34e05efa","Type":"ContainerDied","Data":"e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8"} Sep 05 06:43:18.695106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.695013 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g" event={"ID":"9deef366-9fca-4cde-a2a4-21bf34e05efa","Type":"ContainerDied","Data":"b1d8388438ed039fe66c6d24030e3431d26eb4ad476fd843929d913535c79396"} Sep 05 06:43:18.695106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.695042 2583 scope.go:117] "RemoveContainer" containerID="e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8" Sep 05 06:43:18.706136 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.706115 2583 scope.go:117] "RemoveContainer" containerID="e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8" Sep 05 06:43:18.706405 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:43:18.706385 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8\": container with ID starting with e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8 not found: ID does not exist" containerID="e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8" Sep 05 06:43:18.706498 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.706413 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8"} err="failed to get container status \"e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8\": rpc error: code = NotFound desc = could not find container \"e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8\": container with ID starting with e7df67ccc1e87c90844f32ae179cc9b747d23d69840ccc1aa3c9262420a270e8 not found: ID does not exist" Sep 05 06:43:18.718791 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.718769 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g"] Sep 05 06:43:18.725244 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:18.725222 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a092eb-757f76455-2m96g"] Sep 05 06:43:19.823347 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:19.823307 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9deef366-9fca-4cde-a2a4-21bf34e05efa" path="/var/lib/kubelet/pods/9deef366-9fca-4cde-a2a4-21bf34e05efa/volumes" Sep 05 06:43:25.285894 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:25.285860 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw"] Sep 05 06:43:25.286378 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:25.286127 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" containerID="cri-o://64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d" gracePeriod=30 Sep 05 06:43:30.539244 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.539210 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:43:30.639449 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639373 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-token\") pod \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " Sep 05 06:43:30.639449 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639447 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-data\") pod \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " Sep 05 06:43:30.639695 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639503 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-podinfo\") pod \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " Sep 05 06:43:30.639695 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639539 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-envoy\") pod \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " Sep 05 06:43:30.639695 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639590 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istiod-ca-cert\") pod \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " Sep 05 06:43:30.639695 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639632 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-certs\") pod \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " Sep 05 06:43:30.639695 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639662 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bblt4\" (UniqueName: \"kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-kube-api-access-bblt4\") pod \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " Sep 05 06:43:30.639950 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639697 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-credential-socket\") pod \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " Sep 05 06:43:30.639950 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639790 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-socket\") pod \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\" (UID: \"c3bc3eae-0d6d-47a8-989d-b45774dc3664\") " Sep 05 06:43:30.639950 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639787 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3bc3eae-0d6d-47a8-989d-b45774dc3664" (UID: "c3bc3eae-0d6d-47a8-989d-b45774dc3664"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:43:30.640106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.639995 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3bc3eae-0d6d-47a8-989d-b45774dc3664" (UID: "c3bc3eae-0d6d-47a8-989d-b45774dc3664"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:43:30.640106 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.640069 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3bc3eae-0d6d-47a8-989d-b45774dc3664" (UID: "c3bc3eae-0d6d-47a8-989d-b45774dc3664"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:43:30.640220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.640122 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3bc3eae-0d6d-47a8-989d-b45774dc3664" (UID: "c3bc3eae-0d6d-47a8-989d-b45774dc3664"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:43:30.640220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.640192 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-data\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:30.640220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.640199 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3bc3eae-0d6d-47a8-989d-b45774dc3664" (UID: "c3bc3eae-0d6d-47a8-989d-b45774dc3664"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:43:30.640220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.640209 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:30.640432 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.640223 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-credential-socket\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:30.641754 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.641726 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3bc3eae-0d6d-47a8-989d-b45774dc3664" (UID: "c3bc3eae-0d6d-47a8-989d-b45774dc3664"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:43:30.641882 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.641813 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3bc3eae-0d6d-47a8-989d-b45774dc3664" (UID: "c3bc3eae-0d6d-47a8-989d-b45774dc3664"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:43:30.641882 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.641825 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3bc3eae-0d6d-47a8-989d-b45774dc3664" (UID: "c3bc3eae-0d6d-47a8-989d-b45774dc3664"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 05 06:43:30.642083 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.642064 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-kube-api-access-bblt4" (OuterVolumeSpecName: "kube-api-access-bblt4") pod "c3bc3eae-0d6d-47a8-989d-b45774dc3664" (UID: "c3bc3eae-0d6d-47a8-989d-b45774dc3664"). InnerVolumeSpecName "kube-api-access-bblt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:43:30.741454 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.741420 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istiod-ca-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:30.741454 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.741448 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bblt4\" (UniqueName: \"kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-kube-api-access-bblt4\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:30.741454 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.741459 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-workload-socket\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:30.741694 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.741467 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-token\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:30.741694 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.741497 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-podinfo\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:30.741694 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.741505 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bc3eae-0d6d-47a8-989d-b45774dc3664-istio-envoy\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:43:30.773131 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.773096 2583 generic.go:356] "Generic (PLEG): container finished" podID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerID="64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d" exitCode=0 Sep 05 06:43:30.773269 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.773154 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" Sep 05 06:43:30.773269 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.773183 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" event={"ID":"c3bc3eae-0d6d-47a8-989d-b45774dc3664","Type":"ContainerDied","Data":"64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d"} Sep 05 06:43:30.773269 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.773239 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw" event={"ID":"c3bc3eae-0d6d-47a8-989d-b45774dc3664","Type":"ContainerDied","Data":"917efbf4ea77c7cf760761b01117323bb2046eb4196c2109bcf056a67687827c"} Sep 05 06:43:30.773269 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.773262 2583 scope.go:117] "RemoveContainer" containerID="64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d" Sep 05 06:43:30.782921 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.782904 2583 scope.go:117] "RemoveContainer" containerID="743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5" Sep 05 06:43:30.791223 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.791204 2583 scope.go:117] "RemoveContainer" containerID="64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d" Sep 05 06:43:30.791511 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:43:30.791466 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d\": container with ID starting with 64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d not found: ID does not exist" containerID="64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d" Sep 05 06:43:30.791610 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.791517 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d"} err="failed to get container status \"64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d\": rpc error: code = NotFound desc = could not find container \"64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d\": container with ID starting with 64bb8bb94ac1561ad452ef0c755bca8df8a828ee1c9c1827d9b307d3d0fb6b1d not found: ID does not exist" Sep 05 06:43:30.791610 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.791537 2583 scope.go:117] "RemoveContainer" containerID="743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5" Sep 05 06:43:30.791800 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:43:30.791778 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5\": container with ID starting with 743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5 not found: ID does not exist" containerID="743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5" Sep 05 06:43:30.791857 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.791810 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5"} err="failed to get container status \"743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5\": rpc error: code = NotFound desc = could not find container \"743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5\": container with ID starting with 743fe588b109a549b60377cc56c0d3bbf0a84088a6bb02194375f4b75e035cf5 not found: ID does not exist" Sep 05 06:43:30.798222 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.798199 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw"] Sep 05 06:43:30.802396 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:30.802374 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-a092eb-openshift-default-7c4b7d77dd-hspmw"] Sep 05 06:43:31.822087 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:43:31.822046 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" path="/var/lib/kubelet/pods/c3bc3eae-0d6d-47a8-989d-b45774dc3664/volumes" Sep 05 06:44:11.008581 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.008539 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf"] Sep 05 06:44:11.009118 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.008833 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" podUID="84a075fb-d230-4596-b8fe-9c1458e823a1" containerName="maas-api" containerID="cri-o://01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c" gracePeriod=30 Sep 05 06:44:11.264995 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.264939 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:44:11.416198 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.416169 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrlwn\" (UniqueName: \"kubernetes.io/projected/84a075fb-d230-4596-b8fe-9c1458e823a1-kube-api-access-xrlwn\") pod \"84a075fb-d230-4596-b8fe-9c1458e823a1\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " Sep 05 06:44:11.416365 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.416206 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-metrics-certs\") pod \"84a075fb-d230-4596-b8fe-9c1458e823a1\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " Sep 05 06:44:11.416365 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.416242 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-maas-api-tls\") pod \"84a075fb-d230-4596-b8fe-9c1458e823a1\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " Sep 05 06:44:11.416365 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.416324 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a075fb-d230-4596-b8fe-9c1458e823a1-openshift-service-ca\") pod \"84a075fb-d230-4596-b8fe-9c1458e823a1\" (UID: \"84a075fb-d230-4596-b8fe-9c1458e823a1\") " Sep 05 06:44:11.416743 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.416717 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84a075fb-d230-4596-b8fe-9c1458e823a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "84a075fb-d230-4596-b8fe-9c1458e823a1" (UID: "84a075fb-d230-4596-b8fe-9c1458e823a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:44:11.418336 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.418310 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a075fb-d230-4596-b8fe-9c1458e823a1-kube-api-access-xrlwn" (OuterVolumeSpecName: "kube-api-access-xrlwn") pod "84a075fb-d230-4596-b8fe-9c1458e823a1" (UID: "84a075fb-d230-4596-b8fe-9c1458e823a1"). InnerVolumeSpecName "kube-api-access-xrlwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:44:11.418336 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.418329 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "84a075fb-d230-4596-b8fe-9c1458e823a1" (UID: "84a075fb-d230-4596-b8fe-9c1458e823a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:44:11.418513 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.418395 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "84a075fb-d230-4596-b8fe-9c1458e823a1" (UID: "84a075fb-d230-4596-b8fe-9c1458e823a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:44:11.517568 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.517457 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrlwn\" (UniqueName: \"kubernetes.io/projected/84a075fb-d230-4596-b8fe-9c1458e823a1-kube-api-access-xrlwn\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:44:11.517568 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.517516 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:44:11.517568 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.517525 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a075fb-d230-4596-b8fe-9c1458e823a1-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:44:11.517568 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:11.517535 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a075fb-d230-4596-b8fe-9c1458e823a1-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:44:12.008991 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:12.008955 2583 generic.go:356] "Generic (PLEG): container finished" podID="84a075fb-d230-4596-b8fe-9c1458e823a1" containerID="01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c" exitCode=0 Sep 05 06:44:12.009439 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:12.009005 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" Sep 05 06:44:12.009439 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:12.009039 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" event={"ID":"84a075fb-d230-4596-b8fe-9c1458e823a1","Type":"ContainerDied","Data":"01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c"} Sep 05 06:44:12.009439 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:12.009097 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf" event={"ID":"84a075fb-d230-4596-b8fe-9c1458e823a1","Type":"ContainerDied","Data":"a621a233f1732a2859562018044988b63bd97e9abe5cf22e70e881407398ab6e"} Sep 05 06:44:12.009439 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:12.009119 2583 scope.go:117] "RemoveContainer" containerID="01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c" Sep 05 06:44:12.018516 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:12.018470 2583 scope.go:117] "RemoveContainer" containerID="01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c" Sep 05 06:44:12.018800 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:44:12.018776 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c\": container with ID starting with 01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c not found: ID does not exist" containerID="01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c" Sep 05 06:44:12.018900 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:12.018803 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c"} err="failed to get container status \"01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c\": rpc error: code = NotFound desc = could not find container \"01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c\": container with ID starting with 01e08524245e3571aecf276f4a59e6e304f6231c29a1a4c69c5a896125b3558c not found: ID does not exist" Sep 05 06:44:12.027704 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:12.027683 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf"] Sep 05 06:44:12.032056 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:12.032035 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6efc3a-5d7849dd8f-d6hrf"] Sep 05 06:44:13.822453 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:13.822410 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84a075fb-d230-4596-b8fe-9c1458e823a1" path="/var/lib/kubelet/pods/84a075fb-d230-4596-b8fe-9c1458e823a1/volumes" Sep 05 06:44:29.477152 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.476964 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj"] Sep 05 06:44:29.477841 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.477804 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" podUID="bab9b006-130c-47f1-8d5e-297452e89116" containerName="maas-api" containerID="cri-o://4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b" gracePeriod=30 Sep 05 06:44:29.542023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.541990 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb"] Sep 05 06:44:29.542260 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.542217 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" podUID="cc4aad14-143f-4276-86ce-8f9e2470bf74" containerName="payload-processing" containerID="cri-o://3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459" gracePeriod=30 Sep 05 06:44:29.840556 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.840529 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:44:29.880302 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.880270 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64xtd\" (UniqueName: \"kubernetes.io/projected/cc4aad14-143f-4276-86ce-8f9e2470bf74-kube-api-access-64xtd\") pod \"cc4aad14-143f-4276-86ce-8f9e2470bf74\" (UID: \"cc4aad14-143f-4276-86ce-8f9e2470bf74\") " Sep 05 06:44:29.880471 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.880351 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc4aad14-143f-4276-86ce-8f9e2470bf74-plugins-config-volume\") pod \"cc4aad14-143f-4276-86ce-8f9e2470bf74\" (UID: \"cc4aad14-143f-4276-86ce-8f9e2470bf74\") " Sep 05 06:44:29.880809 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.880778 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc4aad14-143f-4276-86ce-8f9e2470bf74-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cc4aad14-143f-4276-86ce-8f9e2470bf74" (UID: "cc4aad14-143f-4276-86ce-8f9e2470bf74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:44:29.882446 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.882421 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc4aad14-143f-4276-86ce-8f9e2470bf74-kube-api-access-64xtd" (OuterVolumeSpecName: "kube-api-access-64xtd") pod "cc4aad14-143f-4276-86ce-8f9e2470bf74" (UID: "cc4aad14-143f-4276-86ce-8f9e2470bf74"). InnerVolumeSpecName "kube-api-access-64xtd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:44:29.921116 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.921094 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:44:29.980934 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.980908 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bab9b006-130c-47f1-8d5e-297452e89116-openshift-service-ca\") pod \"bab9b006-130c-47f1-8d5e-297452e89116\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " Sep 05 06:44:29.981056 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.980963 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-metrics-certs\") pod \"bab9b006-130c-47f1-8d5e-297452e89116\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " Sep 05 06:44:29.981103 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.981076 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-maas-api-tls\") pod \"bab9b006-130c-47f1-8d5e-297452e89116\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " Sep 05 06:44:29.981150 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.981125 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kf86\" (UniqueName: \"kubernetes.io/projected/bab9b006-130c-47f1-8d5e-297452e89116-kube-api-access-7kf86\") pod \"bab9b006-130c-47f1-8d5e-297452e89116\" (UID: \"bab9b006-130c-47f1-8d5e-297452e89116\") " Sep 05 06:44:29.981310 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.981276 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bab9b006-130c-47f1-8d5e-297452e89116-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bab9b006-130c-47f1-8d5e-297452e89116" (UID: "bab9b006-130c-47f1-8d5e-297452e89116"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:44:29.981423 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.981382 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bab9b006-130c-47f1-8d5e-297452e89116-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:44:29.981423 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.981400 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64xtd\" (UniqueName: \"kubernetes.io/projected/cc4aad14-143f-4276-86ce-8f9e2470bf74-kube-api-access-64xtd\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:44:29.981423 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.981409 2583 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc4aad14-143f-4276-86ce-8f9e2470bf74-plugins-config-volume\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:44:29.983178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.983153 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bab9b006-130c-47f1-8d5e-297452e89116-kube-api-access-7kf86" (OuterVolumeSpecName: "kube-api-access-7kf86") pod "bab9b006-130c-47f1-8d5e-297452e89116" (UID: "bab9b006-130c-47f1-8d5e-297452e89116"). InnerVolumeSpecName "kube-api-access-7kf86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:44:29.983531 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.983507 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bab9b006-130c-47f1-8d5e-297452e89116" (UID: "bab9b006-130c-47f1-8d5e-297452e89116"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:44:29.983628 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:29.983545 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bab9b006-130c-47f1-8d5e-297452e89116" (UID: "bab9b006-130c-47f1-8d5e-297452e89116"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:44:30.082014 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.081937 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:44:30.082014 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.081965 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kf86\" (UniqueName: \"kubernetes.io/projected/bab9b006-130c-47f1-8d5e-297452e89116-kube-api-access-7kf86\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:44:30.082014 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.081980 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bab9b006-130c-47f1-8d5e-297452e89116-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:44:30.117215 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.117184 2583 generic.go:356] "Generic (PLEG): container finished" podID="bab9b006-130c-47f1-8d5e-297452e89116" containerID="4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b" exitCode=0 Sep 05 06:44:30.117353 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.117241 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" Sep 05 06:44:30.117353 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.117278 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" event={"ID":"bab9b006-130c-47f1-8d5e-297452e89116","Type":"ContainerDied","Data":"4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b"} Sep 05 06:44:30.117512 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.117367 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj" event={"ID":"bab9b006-130c-47f1-8d5e-297452e89116","Type":"ContainerDied","Data":"b6981d539346791390be017f507dd6fc6fa2fba30d0f8e1629d5071a93cc9aaf"} Sep 05 06:44:30.117512 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.117402 2583 scope.go:117] "RemoveContainer" containerID="4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b" Sep 05 06:44:30.118795 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.118772 2583 generic.go:356] "Generic (PLEG): container finished" podID="cc4aad14-143f-4276-86ce-8f9e2470bf74" containerID="3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459" exitCode=0 Sep 05 06:44:30.118897 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.118796 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" Sep 05 06:44:30.118897 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.118826 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" event={"ID":"cc4aad14-143f-4276-86ce-8f9e2470bf74","Type":"ContainerDied","Data":"3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459"} Sep 05 06:44:30.118897 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.118894 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb" event={"ID":"cc4aad14-143f-4276-86ce-8f9e2470bf74","Type":"ContainerDied","Data":"28e8efefae4b3bc908c3302abc308050d599e9735d146b245e55fabda57223c6"} Sep 05 06:44:30.130171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.130094 2583 scope.go:117] "RemoveContainer" containerID="4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b" Sep 05 06:44:30.130440 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:44:30.130402 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b\": container with ID starting with 4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b not found: ID does not exist" containerID="4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b" Sep 05 06:44:30.130562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.130435 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b"} err="failed to get container status \"4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b\": rpc error: code = NotFound desc = could not find container \"4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b\": container with ID starting with 4350a84badb659477b611395babacdda9b8684b89bac468072b8ba4e18a0107b not found: ID does not exist" Sep 05 06:44:30.130562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.130457 2583 scope.go:117] "RemoveContainer" containerID="3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459" Sep 05 06:44:30.138739 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.138722 2583 scope.go:117] "RemoveContainer" containerID="3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459" Sep 05 06:44:30.138997 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:44:30.138978 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459\": container with ID starting with 3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459 not found: ID does not exist" containerID="3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459" Sep 05 06:44:30.139038 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.139005 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459"} err="failed to get container status \"3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459\": rpc error: code = NotFound desc = could not find container \"3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459\": container with ID starting with 3839795f78b33ad1b8586f16dc503854da1428c436b21071eeceadd1ba67e459 not found: ID does not exist" Sep 05 06:44:30.148647 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.148625 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb"] Sep 05 06:44:30.150977 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.150953 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5d5f17-d578b8999-mzsnb"] Sep 05 06:44:30.162610 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.162584 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj"] Sep 05 06:44:30.165444 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:30.165422 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5d5f17-5d97f9d4b4-hcqbj"] Sep 05 06:44:31.823659 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:31.823629 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bab9b006-130c-47f1-8d5e-297452e89116" path="/var/lib/kubelet/pods/bab9b006-130c-47f1-8d5e-297452e89116/volumes" Sep 05 06:44:31.824030 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:31.823958 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc4aad14-143f-4276-86ce-8f9e2470bf74" path="/var/lib/kubelet/pods/cc4aad14-143f-4276-86ce-8f9e2470bf74/volumes" Sep 05 06:44:42.282644 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.282604 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5"] Sep 05 06:44:42.283813 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.283349 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bab9b006-130c-47f1-8d5e-297452e89116" containerName="maas-api" Sep 05 06:44:42.283813 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.283376 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bab9b006-130c-47f1-8d5e-297452e89116" containerName="maas-api" Sep 05 06:44:42.283813 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.283444 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a075fb-d230-4596-b8fe-9c1458e823a1" containerName="maas-api" Sep 05 06:44:42.283813 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.283455 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a075fb-d230-4596-b8fe-9c1458e823a1" containerName="maas-api" Sep 05 06:44:42.284171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.283467 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" Sep 05 06:44:42.284327 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284297 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" Sep 05 06:44:42.284469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284331 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbc04aee-bac0-47ec-bf0a-2dd40ce54859" containerName="maas-api" Sep 05 06:44:42.284469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284341 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbc04aee-bac0-47ec-bf0a-2dd40ce54859" containerName="maas-api" Sep 05 06:44:42.284469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284360 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9deef366-9fca-4cde-a2a4-21bf34e05efa" containerName="maas-api" Sep 05 06:44:42.284469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284369 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9deef366-9fca-4cde-a2a4-21bf34e05efa" containerName="maas-api" Sep 05 06:44:42.284469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284387 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc4aad14-143f-4276-86ce-8f9e2470bf74" containerName="payload-processing" Sep 05 06:44:42.284469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284397 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc4aad14-143f-4276-86ce-8f9e2470bf74" containerName="payload-processing" Sep 05 06:44:42.284469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284416 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" Sep 05 06:44:42.284469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284427 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" Sep 05 06:44:42.284884 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284573 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" Sep 05 06:44:42.284884 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284585 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc4aad14-143f-4276-86ce-8f9e2470bf74" containerName="payload-processing" Sep 05 06:44:42.284884 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284593 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a075fb-d230-4596-b8fe-9c1458e823a1" containerName="maas-api" Sep 05 06:44:42.284884 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284603 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbc04aee-bac0-47ec-bf0a-2dd40ce54859" containerName="maas-api" Sep 05 06:44:42.284884 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284610 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bab9b006-130c-47f1-8d5e-297452e89116" containerName="maas-api" Sep 05 06:44:42.284884 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.284616 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9deef366-9fca-4cde-a2a4-21bf34e05efa" containerName="maas-api" Sep 05 06:44:42.288249 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.288230 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.291057 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.291036 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-ef54fd-serving-cert\"" Sep 05 06:44:42.298207 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.298183 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5"] Sep 05 06:44:42.393191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.393147 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.393374 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.393260 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/598a193a-d4ee-4295-92cc-6c65e4044830-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.393437 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.393399 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.393509 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.393454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxfz8\" (UniqueName: \"kubernetes.io/projected/598a193a-d4ee-4295-92cc-6c65e4044830-kube-api-access-pxfz8\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.494830 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.494789 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/598a193a-d4ee-4295-92cc-6c65e4044830-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.495007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.494896 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.495007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.494940 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxfz8\" (UniqueName: \"kubernetes.io/projected/598a193a-d4ee-4295-92cc-6c65e4044830-kube-api-access-pxfz8\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.495007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.494975 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.495437 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.495415 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/598a193a-d4ee-4295-92cc-6c65e4044830-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.497353 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.497329 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.497441 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.497381 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.503739 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.503713 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxfz8\" (UniqueName: \"kubernetes.io/projected/598a193a-d4ee-4295-92cc-6c65e4044830-kube-api-access-pxfz8\") pod \"maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.600400 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.600316 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:42.880510 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:42.880466 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5"] Sep 05 06:44:42.880928 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:44:42.880903 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598a193a_d4ee_4295_92cc_6c65e4044830.slice/crio-3eb6394998a409877c2c8eeb68f82bd5bfe2e5bb181c805598ea05a08459a159 WatchSource:0}: Error finding container 3eb6394998a409877c2c8eeb68f82bd5bfe2e5bb181c805598ea05a08459a159: Status 404 returned error can't find the container with id 3eb6394998a409877c2c8eeb68f82bd5bfe2e5bb181c805598ea05a08459a159 Sep 05 06:44:43.197629 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:43.197548 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" event={"ID":"598a193a-d4ee-4295-92cc-6c65e4044830","Type":"ContainerStarted","Data":"3eb6394998a409877c2c8eeb68f82bd5bfe2e5bb181c805598ea05a08459a159"} Sep 05 06:44:44.204455 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:44.204415 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" event={"ID":"598a193a-d4ee-4295-92cc-6c65e4044830","Type":"ContainerStarted","Data":"6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56"} Sep 05 06:44:44.204876 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:44.204511 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:44.225360 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:44.225309 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" podStartSLOduration=1.6188834669999999 podStartE2EDuration="2.22529688s" totalImagesPullingTime="606.413413ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:44:44.223365112 +0000 UTC m=+1180.906870031" watchObservedRunningTime="2026-09-05 06:44:44.22529688 +0000 UTC m=+1180.908801787" Sep 05 06:44:50.215249 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:50.215215 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:44:53.483067 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.483025 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp"] Sep 05 06:44:53.483807 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.483788 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3bc3eae-0d6d-47a8-989d-b45774dc3664" containerName="istio-proxy" Sep 05 06:44:53.487988 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.487585 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.490400 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.490378 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-01e18c68-serving-cert\"" Sep 05 06:44:53.499338 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.499314 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp"] Sep 05 06:44:53.594924 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.594888 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-maas-api-tls\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.595104 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.594945 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-metrics-certs\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.595104 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.594980 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-openshift-service-ca\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.595104 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.595094 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44psm\" (UniqueName: \"kubernetes.io/projected/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-kube-api-access-44psm\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.695825 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.695795 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-maas-api-tls\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.695971 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.695839 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-metrics-certs\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.695971 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.695861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-openshift-service-ca\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.695971 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.695916 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44psm\" (UniqueName: \"kubernetes.io/projected/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-kube-api-access-44psm\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.696614 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.696589 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-openshift-service-ca\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.698340 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.698316 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-maas-api-tls\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.698427 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.698363 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-metrics-certs\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.707088 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.707065 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44psm\" (UniqueName: \"kubernetes.io/projected/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-kube-api-access-44psm\") pod \"maas-api-e2e-mt-01e18c68-89cbcf679-qlljp\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:53.801000 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:53.800933 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:54.079327 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:54.079298 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp"] Sep 05 06:44:54.080347 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:44:54.080319 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ac65f6_6f9a_4000_85fb_6f6afef6bac3.slice/crio-8a4050a82ffec51ff4a9c5cbabefcc9e940c9e0ffb0e6b0901d942b0656a3e03 WatchSource:0}: Error finding container 8a4050a82ffec51ff4a9c5cbabefcc9e940c9e0ffb0e6b0901d942b0656a3e03: Status 404 returned error can't find the container with id 8a4050a82ffec51ff4a9c5cbabefcc9e940c9e0ffb0e6b0901d942b0656a3e03 Sep 05 06:44:54.273164 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:54.273117 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" event={"ID":"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3","Type":"ContainerStarted","Data":"8a4050a82ffec51ff4a9c5cbabefcc9e940c9e0ffb0e6b0901d942b0656a3e03"} Sep 05 06:44:55.281155 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:55.281111 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" event={"ID":"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3","Type":"ContainerStarted","Data":"f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5"} Sep 05 06:44:55.281728 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:55.281217 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:44:55.303178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:44:55.303121 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" podStartSLOduration=1.908836676 podStartE2EDuration="2.30310626s" totalImagesPullingTime="394.269584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:44:55.301157433 +0000 UTC m=+1191.984662340" watchObservedRunningTime="2026-09-05 06:44:55.30310626 +0000 UTC m=+1191.986611167" Sep 05 06:45:01.294082 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:01.294051 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:45:03.873315 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:03.873276 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:45:03.880262 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:03.880236 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:45:03.882791 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:03.882768 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:45:03.888612 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:03.888593 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:45:06.755319 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.755283 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr"] Sep 05 06:45:06.760907 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.760885 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:06.763265 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.763241 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-5b1d24-openshift-default-dockercfg-xt6vz\"" Sep 05 06:45:06.772010 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.771978 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr"] Sep 05 06:45:06.923110 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.923067 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-podinfo\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:06.923290 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.923185 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-credential-socket\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:06.923290 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.923214 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-token\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:06.923290 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.923237 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-data\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:06.923290 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.923261 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istiod-ca-cert\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:06.923290 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.923281 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-certs\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:06.923519 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.923344 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhjrk\" (UniqueName: \"kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-kube-api-access-qhjrk\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:06.923519 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.923382 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-envoy\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:06.923519 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:06.923427 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-socket\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.024823 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.024731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-credential-socket\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.024823 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.024767 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-token\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.024823 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.024796 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-data\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.024823 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.024827 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istiod-ca-cert\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.025176 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.024852 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-certs\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.025176 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.024877 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhjrk\" (UniqueName: \"kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-kube-api-access-qhjrk\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.025176 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.024906 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-envoy\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.025176 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.024944 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-socket\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.025176 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.025009 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-podinfo\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.025399 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.025265 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-data\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.025399 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.025319 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-socket\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.025399 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.025347 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-certs\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.025547 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.025459 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-credential-socket\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.025622 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.025604 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istiod-ca-cert\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.027297 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.027275 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-envoy\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.027627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.027605 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-podinfo\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.033248 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.033223 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhjrk\" (UniqueName: \"kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-kube-api-access-qhjrk\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.033426 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.033405 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-token\") pod \"e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.075795 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.075760 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:07.355897 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.355871 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr"] Sep 05 06:45:07.357444 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:45:07.357415 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb5ad39_4ffa_42d1_aca3_44839d5f52ff.slice/crio-70af58974953536e481660fe26b6a0dd1195323aa3580e52d7233c64360509de WatchSource:0}: Error finding container 70af58974953536e481660fe26b6a0dd1195323aa3580e52d7233c64360509de: Status 404 returned error can't find the container with id 70af58974953536e481660fe26b6a0dd1195323aa3580e52d7233c64360509de Sep 05 06:45:07.359773 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.359736 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:45:07.359873 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.359808 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:45:07.359873 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:07.359841 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:45:08.363395 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:08.363350 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" event={"ID":"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff","Type":"ContainerStarted","Data":"c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab"} Sep 05 06:45:08.363395 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:08.363402 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" event={"ID":"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff","Type":"ContainerStarted","Data":"70af58974953536e481660fe26b6a0dd1195323aa3580e52d7233c64360509de"} Sep 05 06:45:09.076409 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:09.076373 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:09.081387 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:09.081368 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:09.111392 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:09.111342 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" podStartSLOduration=3.111327171 podStartE2EDuration="3.111327171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:45:08.387387912 +0000 UTC m=+1205.070892832" watchObservedRunningTime="2026-09-05 06:45:09.111327171 +0000 UTC m=+1205.794832078" Sep 05 06:45:09.369804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:09.369714 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:09.370818 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:09.370797 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:11.520111 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.520076 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8"] Sep 05 06:45:11.524343 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.524320 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.526813 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.526744 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-5b1d24-serving-cert\"" Sep 05 06:45:11.535785 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.535760 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8"] Sep 05 06:45:11.673735 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.673694 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.673925 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.673781 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8089fd90-d284-4828-8d60-bd5f6d49065a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.673925 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.673820 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.673925 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.673845 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsz8s\" (UniqueName: \"kubernetes.io/projected/8089fd90-d284-4828-8d60-bd5f6d49065a-kube-api-access-dsz8s\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.775254 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.775152 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.775433 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.775255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8089fd90-d284-4828-8d60-bd5f6d49065a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.775433 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.775337 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.775433 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.775365 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsz8s\" (UniqueName: \"kubernetes.io/projected/8089fd90-d284-4828-8d60-bd5f6d49065a-kube-api-access-dsz8s\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.776050 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.776027 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8089fd90-d284-4828-8d60-bd5f6d49065a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.777840 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.777806 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.777936 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.777877 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.786846 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.786826 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsz8s\" (UniqueName: \"kubernetes.io/projected/8089fd90-d284-4828-8d60-bd5f6d49065a-kube-api-access-dsz8s\") pod \"maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:11.840936 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:11.840911 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:12.117716 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:12.117691 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8"] Sep 05 06:45:12.119497 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:45:12.119457 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8089fd90_d284_4828_8d60_bd5f6d49065a.slice/crio-b5d51132401cca5512b6b3fbaa111674e57c5cd1b0ff7fc321e9bcdeba04ead1 WatchSource:0}: Error finding container b5d51132401cca5512b6b3fbaa111674e57c5cd1b0ff7fc321e9bcdeba04ead1: Status 404 returned error can't find the container with id b5d51132401cca5512b6b3fbaa111674e57c5cd1b0ff7fc321e9bcdeba04ead1 Sep 05 06:45:12.388041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:12.387950 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" event={"ID":"8089fd90-d284-4828-8d60-bd5f6d49065a","Type":"ContainerStarted","Data":"b5d51132401cca5512b6b3fbaa111674e57c5cd1b0ff7fc321e9bcdeba04ead1"} Sep 05 06:45:13.396214 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:13.396176 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" event={"ID":"8089fd90-d284-4828-8d60-bd5f6d49065a","Type":"ContainerStarted","Data":"e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6"} Sep 05 06:45:13.396631 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:13.396259 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:13.417901 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:13.417851 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" podStartSLOduration=2.035575936 podStartE2EDuration="2.417835683s" totalImagesPullingTime="382.259747ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:45:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:45:13.417053757 +0000 UTC m=+1210.100558665" watchObservedRunningTime="2026-09-05 06:45:13.417835683 +0000 UTC m=+1210.101340587" Sep 05 06:45:19.426774 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:19.426740 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:45:25.411651 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:25.411604 2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 05 06:45:25.412057 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:25.411672 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 05 06:45:26.478285 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:26.478251 2583 generic.go:356] "Generic (PLEG): container finished" podID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerID="c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab" exitCode=0 Sep 05 06:45:26.478740 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:26.478324 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" event={"ID":"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff","Type":"ContainerDied","Data":"c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab"} Sep 05 06:45:26.478789 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:26.478756 2583 scope.go:117] "RemoveContainer" containerID="c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab" Sep 05 06:45:26.480506 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:26.480442 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:45:26.480660 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:26.480636 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:45:26.480921 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:26.480894 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 05 06:45:27.486107 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:27.486067 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" event={"ID":"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff","Type":"ContainerStarted","Data":"ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb"} Sep 05 06:45:28.075913 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:28.075880 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:28.076083 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:28.076001 2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Sep 05 06:45:28.076083 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:28.076053 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Sep 05 06:45:29.076922 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:29.076883 2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Sep 05 06:45:29.077424 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:29.076955 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Sep 05 06:45:30.076497 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:30.076440 2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Sep 05 06:45:30.076674 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:30.076519 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Sep 05 06:45:31.076707 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:31.076675 2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Sep 05 06:45:31.077087 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:31.076729 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Sep 05 06:45:32.076689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:32.076644 2583 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Sep 05 06:45:32.076891 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:32.076709 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Sep 05 06:45:33.080192 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:33.080158 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:33.080687 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:33.080424 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:33.081070 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:33.081050 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:45:35.663687 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:35.663653 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp"] Sep 05 06:45:35.664131 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:35.663869 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" podUID="f3ac65f6-6f9a-4000-85fb-6f6afef6bac3" containerName="maas-api" containerID="cri-o://f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5" gracePeriod=30 Sep 05 06:45:35.923484 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:35.923418 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:45:36.020693 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.020658 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-metrics-certs\") pod \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " Sep 05 06:45:36.020839 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.020716 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44psm\" (UniqueName: \"kubernetes.io/projected/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-kube-api-access-44psm\") pod \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " Sep 05 06:45:36.020839 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.020789 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-openshift-service-ca\") pod \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " Sep 05 06:45:36.020919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.020904 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-maas-api-tls\") pod \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\" (UID: \"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3\") " Sep 05 06:45:36.021203 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.021170 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3ac65f6-6f9a-4000-85fb-6f6afef6bac3" (UID: "f3ac65f6-6f9a-4000-85fb-6f6afef6bac3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:45:36.021325 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.021300 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:45:36.022778 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.022754 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3ac65f6-6f9a-4000-85fb-6f6afef6bac3" (UID: "f3ac65f6-6f9a-4000-85fb-6f6afef6bac3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:45:36.022888 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.022847 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f3ac65f6-6f9a-4000-85fb-6f6afef6bac3" (UID: "f3ac65f6-6f9a-4000-85fb-6f6afef6bac3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:45:36.022950 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.022936 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-kube-api-access-44psm" (OuterVolumeSpecName: "kube-api-access-44psm") pod "f3ac65f6-6f9a-4000-85fb-6f6afef6bac3" (UID: "f3ac65f6-6f9a-4000-85fb-6f6afef6bac3"). InnerVolumeSpecName "kube-api-access-44psm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:45:36.122497 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.122433 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:45:36.122497 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.122499 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44psm\" (UniqueName: \"kubernetes.io/projected/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-kube-api-access-44psm\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:45:36.122497 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.122510 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:45:36.548228 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.548191 2583 generic.go:356] "Generic (PLEG): container finished" podID="f3ac65f6-6f9a-4000-85fb-6f6afef6bac3" containerID="f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5" exitCode=0 Sep 05 06:45:36.548438 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.548229 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" event={"ID":"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3","Type":"ContainerDied","Data":"f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5"} Sep 05 06:45:36.548438 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.548296 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" event={"ID":"f3ac65f6-6f9a-4000-85fb-6f6afef6bac3","Type":"ContainerDied","Data":"8a4050a82ffec51ff4a9c5cbabefcc9e940c9e0ffb0e6b0901d942b0656a3e03"} Sep 05 06:45:36.548438 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.548239 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp" Sep 05 06:45:36.548438 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.548329 2583 scope.go:117] "RemoveContainer" containerID="f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5" Sep 05 06:45:36.558087 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.558071 2583 scope.go:117] "RemoveContainer" containerID="f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5" Sep 05 06:45:36.558383 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:45:36.558362 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5\": container with ID starting with f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5 not found: ID does not exist" containerID="f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5" Sep 05 06:45:36.558448 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.558391 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5"} err="failed to get container status \"f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5\": rpc error: code = NotFound desc = could not find container \"f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5\": container with ID starting with f8303b0f185863bbad23d6d9880a043f60bd6c131d62b587f2bed7295e845ca5 not found: ID does not exist" Sep 05 06:45:36.571907 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.571886 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp"] Sep 05 06:45:36.577655 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:36.577632 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-01e18c68-89cbcf679-qlljp"] Sep 05 06:45:37.823903 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:45:37.823862 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3ac65f6-6f9a-4000-85fb-6f6afef6bac3" path="/var/lib/kubelet/pods/f3ac65f6-6f9a-4000-85fb-6f6afef6bac3/volumes" Sep 05 06:46:01.858871 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.858785 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4"] Sep 05 06:46:01.859538 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.859515 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3ac65f6-6f9a-4000-85fb-6f6afef6bac3" containerName="maas-api" Sep 05 06:46:01.859538 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.859539 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3ac65f6-6f9a-4000-85fb-6f6afef6bac3" containerName="maas-api" Sep 05 06:46:01.859743 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.859663 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3ac65f6-6f9a-4000-85fb-6f6afef6bac3" containerName="maas-api" Sep 05 06:46:01.864764 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.864743 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:01.868083 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.867713 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-eac764-serving-cert\"" Sep 05 06:46:01.870635 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.870608 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4"] Sep 05 06:46:01.974036 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.973992 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:01.974235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.974069 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89fsc\" (UniqueName: \"kubernetes.io/projected/7d788851-50b3-40f1-8299-de8adde55bb0-kube-api-access-89fsc\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:01.974235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.974130 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:01.974235 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:01.974158 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d788851-50b3-40f1-8299-de8adde55bb0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:02.075349 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.075309 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:02.075545 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.075366 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89fsc\" (UniqueName: \"kubernetes.io/projected/7d788851-50b3-40f1-8299-de8adde55bb0-kube-api-access-89fsc\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:02.075545 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.075405 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:02.075545 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.075429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d788851-50b3-40f1-8299-de8adde55bb0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:02.076224 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.076199 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d788851-50b3-40f1-8299-de8adde55bb0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:02.077852 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.077832 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:02.077974 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.077953 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:02.085767 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.085738 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89fsc\" (UniqueName: \"kubernetes.io/projected/7d788851-50b3-40f1-8299-de8adde55bb0-kube-api-access-89fsc\") pod \"maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:02.178104 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.178012 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:02.459738 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.459708 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4"] Sep 05 06:46:02.707534 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:02.707495 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" event={"ID":"7d788851-50b3-40f1-8299-de8adde55bb0","Type":"ContainerStarted","Data":"28dcc836ca39bd6108814932f3089a7c0ff212d2a61db2f09f318b42999abaa6"} Sep 05 06:46:03.714853 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:03.714817 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" event={"ID":"7d788851-50b3-40f1-8299-de8adde55bb0","Type":"ContainerStarted","Data":"829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a"} Sep 05 06:46:03.715245 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:03.714933 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:03.734350 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:03.734301 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" podStartSLOduration=2.217234862 podStartE2EDuration="2.734285209s" totalImagesPullingTime="517.050347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:46:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:46:03.732750819 +0000 UTC m=+1260.416255726" watchObservedRunningTime="2026-09-05 06:46:03.734285209 +0000 UTC m=+1260.417790513" Sep 05 06:46:09.725274 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:09.725244 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:46:30.963653 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:30.963618 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp"] Sep 05 06:46:30.967429 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:30.967406 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:30.970534 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:30.970508 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-32e859-serving-cert\"" Sep 05 06:46:30.978997 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:30.978972 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp"] Sep 05 06:46:31.059676 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.059640 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9748a856-4932-4830-9e9b-9564e9402e45-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.059676 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.059680 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.059927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.059790 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fws5t\" (UniqueName: \"kubernetes.io/projected/9748a856-4932-4830-9e9b-9564e9402e45-kube-api-access-fws5t\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.059927 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.059880 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.161209 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.161168 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9748a856-4932-4830-9e9b-9564e9402e45-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.161209 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.161209 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.161453 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.161253 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fws5t\" (UniqueName: \"kubernetes.io/projected/9748a856-4932-4830-9e9b-9564e9402e45-kube-api-access-fws5t\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.161453 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.161286 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.161956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.161932 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9748a856-4932-4830-9e9b-9564e9402e45-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.163696 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.163667 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.163807 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.163771 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.169434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.169414 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fws5t\" (UniqueName: \"kubernetes.io/projected/9748a856-4932-4830-9e9b-9564e9402e45-kube-api-access-fws5t\") pod \"maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.285185 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.285150 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:31.587523 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:46:31.587491 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9748a856_4932_4830_9e9b_9564e9402e45.slice/crio-dce45c8c6dc5fc2e7eaf2e7065c76dbef706759e707c5152d3be39c2bb2a411a WatchSource:0}: Error finding container dce45c8c6dc5fc2e7eaf2e7065c76dbef706759e707c5152d3be39c2bb2a411a: Status 404 returned error can't find the container with id dce45c8c6dc5fc2e7eaf2e7065c76dbef706759e707c5152d3be39c2bb2a411a Sep 05 06:46:31.587667 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.587624 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp"] Sep 05 06:46:31.889563 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:31.889453 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" event={"ID":"9748a856-4932-4830-9e9b-9564e9402e45","Type":"ContainerStarted","Data":"dce45c8c6dc5fc2e7eaf2e7065c76dbef706759e707c5152d3be39c2bb2a411a"} Sep 05 06:46:32.896443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:32.896403 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" event={"ID":"9748a856-4932-4830-9e9b-9564e9402e45","Type":"ContainerStarted","Data":"aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab"} Sep 05 06:46:32.896841 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:32.896552 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:46:32.921306 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:32.921263 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" podStartSLOduration=2.480893633 podStartE2EDuration="2.921246983s" totalImagesPullingTime="440.35335ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:46:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:46:32.918172143 +0000 UTC m=+1289.601677051" watchObservedRunningTime="2026-09-05 06:46:32.921246983 +0000 UTC m=+1289.604751890" Sep 05 06:46:38.906495 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:46:38.906441 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:47:12.190919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.190832 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5"] Sep 05 06:47:12.195337 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.195307 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.197845 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.197817 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c36aacf1-serving-cert\"" Sep 05 06:47:12.207506 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.207463 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5"] Sep 05 06:47:12.362844 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.362809 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-metrics-certs\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.362844 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.362841 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-maas-api-tls\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.363092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.362864 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-485zg\" (UniqueName: \"kubernetes.io/projected/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-kube-api-access-485zg\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.363092 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.363041 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-openshift-service-ca\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.463691 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.463603 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-metrics-certs\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.463691 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.463640 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-maas-api-tls\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.463691 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.463663 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-485zg\" (UniqueName: \"kubernetes.io/projected/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-kube-api-access-485zg\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.463970 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.463736 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-openshift-service-ca\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.463970 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:47:12.463777 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-serving-cert: secret "maas-api-e2e-mt-c36aacf1-serving-cert" not found Sep 05 06:47:12.463970 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:47:12.463869 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-maas-api-tls podName:ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8 nodeName:}" failed. No retries permitted until 2026-09-05 06:47:12.96384807 +0000 UTC m=+1329.647352963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-maas-api-tls") pod "maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" (UID: "ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8") : secret "maas-api-e2e-mt-c36aacf1-serving-cert" not found Sep 05 06:47:12.464409 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.464387 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-openshift-service-ca\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.466023 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.465994 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-metrics-certs\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.472825 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.472800 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-485zg\" (UniqueName: \"kubernetes.io/projected/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-kube-api-access-485zg\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.969178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.969134 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-maas-api-tls\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:12.972038 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:12.972006 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-maas-api-tls\") pod \"maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:13.109002 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:13.108967 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:13.385169 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:13.385071 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5"] Sep 05 06:47:14.152209 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:14.152114 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" event={"ID":"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8","Type":"ContainerStarted","Data":"7f503f8dea388a04a9d3c4d187fe1b285e6a89adde22f594703cf9df3dec2259"} Sep 05 06:47:14.152454 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:14.152426 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:47:14.152592 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:14.152467 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" event={"ID":"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8","Type":"ContainerStarted","Data":"030c0df95b6c23bb6e372048781200e899b37a20fb5ecd048e5c79fe83b6067d"} Sep 05 06:47:14.174055 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:14.174004 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" podStartSLOduration=1.724876276 podStartE2EDuration="2.173988158s" totalImagesPullingTime="449.111882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:47:14.171756509 +0000 UTC m=+1330.855261416" watchObservedRunningTime="2026-09-05 06:47:14.173988158 +0000 UTC m=+1330.857493070" Sep 05 06:47:20.164204 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:47:20.164174 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:48:36.426124 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.426090 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5"] Sep 05 06:48:36.426708 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.426368 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" podUID="ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8" containerName="maas-api" containerID="cri-o://7f503f8dea388a04a9d3c4d187fe1b285e6a89adde22f594703cf9df3dec2259" gracePeriod=30 Sep 05 06:48:36.655647 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.655615 2583 generic.go:356] "Generic (PLEG): container finished" podID="ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8" containerID="7f503f8dea388a04a9d3c4d187fe1b285e6a89adde22f594703cf9df3dec2259" exitCode=0 Sep 05 06:48:36.655809 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.655685 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" event={"ID":"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8","Type":"ContainerDied","Data":"7f503f8dea388a04a9d3c4d187fe1b285e6a89adde22f594703cf9df3dec2259"} Sep 05 06:48:36.686767 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.686712 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:48:36.775083 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.775050 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-maas-api-tls\") pod \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " Sep 05 06:48:36.775258 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.775104 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-openshift-service-ca\") pod \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " Sep 05 06:48:36.775258 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.775149 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-metrics-certs\") pod \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " Sep 05 06:48:36.775258 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.775238 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-485zg\" (UniqueName: \"kubernetes.io/projected/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-kube-api-access-485zg\") pod \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\" (UID: \"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8\") " Sep 05 06:48:36.775707 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.775674 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8" (UID: "ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:48:36.777527 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.777432 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-kube-api-access-485zg" (OuterVolumeSpecName: "kube-api-access-485zg") pod "ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8" (UID: "ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8"). InnerVolumeSpecName "kube-api-access-485zg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:48:36.777527 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.777496 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8" (UID: "ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:48:36.777707 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.777559 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8" (UID: "ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:48:36.876949 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.876903 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-485zg\" (UniqueName: \"kubernetes.io/projected/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-kube-api-access-485zg\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:48:36.876949 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.876933 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:48:36.876949 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.876942 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:48:36.876949 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:36.876951 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:48:37.664056 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:37.664011 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" event={"ID":"ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8","Type":"ContainerDied","Data":"030c0df95b6c23bb6e372048781200e899b37a20fb5ecd048e5c79fe83b6067d"} Sep 05 06:48:37.664541 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:37.664030 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5" Sep 05 06:48:37.664541 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:37.664117 2583 scope.go:117] "RemoveContainer" containerID="7f503f8dea388a04a9d3c4d187fe1b285e6a89adde22f594703cf9df3dec2259" Sep 05 06:48:37.688229 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:37.688208 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5"] Sep 05 06:48:37.694327 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:37.694303 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c36aacf1-7587d7b687-9fsp5"] Sep 05 06:48:37.823433 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:48:37.823394 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8" path="/var/lib/kubelet/pods/ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8/volumes" Sep 05 06:50:03.923566 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:03.923536 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:50:03.929752 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:03.929729 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:50:03.931565 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:03.931547 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:50:03.937384 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:03.937363 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:50:29.236898 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.236865 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb"] Sep 05 06:50:29.239342 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.237319 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8" containerName="maas-api" Sep 05 06:50:29.239342 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.237330 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8" containerName="maas-api" Sep 05 06:50:29.239342 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.237399 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecf8b1a0-2d47-4ca6-afc6-75723e1e86f8" containerName="maas-api" Sep 05 06:50:29.240511 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.240488 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.243156 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.243129 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-73c192-serving-cert\"" Sep 05 06:50:29.252657 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.252634 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb"] Sep 05 06:50:29.312744 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.312709 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4r5t\" (UniqueName: \"kubernetes.io/projected/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-kube-api-access-g4r5t\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.312919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.312774 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-maas-api-tls\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.312919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.312854 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-metrics-certs\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.312919 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.312907 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-openshift-service-ca\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.414452 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.414413 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-metrics-certs\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.414452 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.414452 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-openshift-service-ca\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.414712 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.414547 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4r5t\" (UniqueName: \"kubernetes.io/projected/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-kube-api-access-g4r5t\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.414712 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.414588 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-maas-api-tls\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.415155 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.415131 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-openshift-service-ca\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.417040 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.417015 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-metrics-certs\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.417127 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.417084 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-maas-api-tls\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.422370 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.422349 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4r5t\" (UniqueName: \"kubernetes.io/projected/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-kube-api-access-g4r5t\") pod \"maas-api-e2e-api-a-73c192-bf5844755-7c5cb\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.553144 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.553058 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:29.833045 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.833021 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb"] Sep 05 06:50:29.835897 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:50:29.835872 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85f1ce5_0da7_4760_935e_bdb60ca1c7de.slice/crio-c7417aafe844cd05c292db081891e6833e6a103cb03bac540ab91f552d37eb7a WatchSource:0}: Error finding container c7417aafe844cd05c292db081891e6833e6a103cb03bac540ab91f552d37eb7a: Status 404 returned error can't find the container with id c7417aafe844cd05c292db081891e6833e6a103cb03bac540ab91f552d37eb7a Sep 05 06:50:29.837249 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:29.837228 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 05 06:50:30.333697 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:30.333660 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" event={"ID":"e85f1ce5-0da7-4760-935e-bdb60ca1c7de","Type":"ContainerStarted","Data":"c7417aafe844cd05c292db081891e6833e6a103cb03bac540ab91f552d37eb7a"} Sep 05 06:50:31.345248 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:31.345208 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" event={"ID":"e85f1ce5-0da7-4760-935e-bdb60ca1c7de","Type":"ContainerStarted","Data":"d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b"} Sep 05 06:50:31.345772 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:31.345393 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:31.363206 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:31.363157 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" podStartSLOduration=1.798467507 podStartE2EDuration="2.363140564s" totalImagesPullingTime="564.673057ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:50:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:50:31.361128972 +0000 UTC m=+1528.044633875" watchObservedRunningTime="2026-09-05 06:50:31.363140564 +0000 UTC m=+1528.046645518" Sep 05 06:50:37.356534 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:37.356500 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:50:58.429451 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.429413 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m"] Sep 05 06:50:58.433940 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.433673 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.436415 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.436390 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-d91c81-serving-cert\"" Sep 05 06:50:58.447750 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.447727 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m"] Sep 05 06:50:58.498173 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.498139 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87306c74-150b-4177-b2f3-f1421cc7b56e-openshift-service-ca\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.498347 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.498185 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-maas-api-tls\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.498347 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.498271 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8skcr\" (UniqueName: \"kubernetes.io/projected/87306c74-150b-4177-b2f3-f1421cc7b56e-kube-api-access-8skcr\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.498440 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.498373 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-metrics-certs\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.599238 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.599205 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-metrics-certs\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.599429 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.599254 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87306c74-150b-4177-b2f3-f1421cc7b56e-openshift-service-ca\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.599429 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.599378 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-maas-api-tls\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.599575 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.599460 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8skcr\" (UniqueName: \"kubernetes.io/projected/87306c74-150b-4177-b2f3-f1421cc7b56e-kube-api-access-8skcr\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.599575 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:50:58.599536 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-serving-cert: secret "maas-api-e2e-api-b-d91c81-serving-cert" not found Sep 05 06:50:58.599700 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:50:58.599640 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-maas-api-tls podName:87306c74-150b-4177-b2f3-f1421cc7b56e nodeName:}" failed. No retries permitted until 2026-09-05 06:50:59.099603748 +0000 UTC m=+1555.783108652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-maas-api-tls") pod "maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" (UID: "87306c74-150b-4177-b2f3-f1421cc7b56e") : secret "maas-api-e2e-api-b-d91c81-serving-cert" not found Sep 05 06:50:58.599986 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.599967 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87306c74-150b-4177-b2f3-f1421cc7b56e-openshift-service-ca\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.601737 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.601714 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-metrics-certs\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:58.608273 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:58.608246 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8skcr\" (UniqueName: \"kubernetes.io/projected/87306c74-150b-4177-b2f3-f1421cc7b56e-kube-api-access-8skcr\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:59.104143 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:59.104108 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-maas-api-tls\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:59.106915 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:59.106883 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-maas-api-tls\") pod \"maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:59.346447 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:59.346414 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:50:59.619434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:50:59.619410 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m"] Sep 05 06:50:59.620745 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:50:59.620719 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87306c74_150b_4177_b2f3_f1421cc7b56e.slice/crio-a410e7a16e0bfe67b8a98d60c3ccf6a13a95b04dc18664ab4ca5b9fe9e7c4661 WatchSource:0}: Error finding container a410e7a16e0bfe67b8a98d60c3ccf6a13a95b04dc18664ab4ca5b9fe9e7c4661: Status 404 returned error can't find the container with id a410e7a16e0bfe67b8a98d60c3ccf6a13a95b04dc18664ab4ca5b9fe9e7c4661 Sep 05 06:51:00.526824 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:00.526786 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" event={"ID":"87306c74-150b-4177-b2f3-f1421cc7b56e","Type":"ContainerStarted","Data":"68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c"} Sep 05 06:51:00.527041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:00.526874 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" event={"ID":"87306c74-150b-4177-b2f3-f1421cc7b56e","Type":"ContainerStarted","Data":"a410e7a16e0bfe67b8a98d60c3ccf6a13a95b04dc18664ab4ca5b9fe9e7c4661"} Sep 05 06:51:00.527041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:00.527009 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:51:00.545147 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:00.545100 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" podStartSLOduration=2.207898096 podStartE2EDuration="2.545084992s" totalImagesPullingTime="337.186896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:50:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:51:00.542940926 +0000 UTC m=+1557.226445832" watchObservedRunningTime="2026-09-05 06:51:00.545084992 +0000 UTC m=+1557.228589898" Sep 05 06:51:06.538933 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:06.538902 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:51:41.035908 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.035877 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb"] Sep 05 06:51:41.036455 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.036093 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" podUID="e85f1ce5-0da7-4760-935e-bdb60ca1c7de" containerName="maas-api" containerID="cri-o://d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b" gracePeriod=30 Sep 05 06:51:41.298013 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.297991 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:51:41.314036 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.314007 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-metrics-certs\") pod \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " Sep 05 06:51:41.314158 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.314053 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4r5t\" (UniqueName: \"kubernetes.io/projected/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-kube-api-access-g4r5t\") pod \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " Sep 05 06:51:41.314158 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.314095 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-maas-api-tls\") pod \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " Sep 05 06:51:41.314158 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.314135 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-openshift-service-ca\") pod \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\" (UID: \"e85f1ce5-0da7-4760-935e-bdb60ca1c7de\") " Sep 05 06:51:41.314550 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.314510 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e85f1ce5-0da7-4760-935e-bdb60ca1c7de" (UID: "e85f1ce5-0da7-4760-935e-bdb60ca1c7de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:51:41.316089 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.316065 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e85f1ce5-0da7-4760-935e-bdb60ca1c7de" (UID: "e85f1ce5-0da7-4760-935e-bdb60ca1c7de"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:51:41.316275 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.316249 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e85f1ce5-0da7-4760-935e-bdb60ca1c7de" (UID: "e85f1ce5-0da7-4760-935e-bdb60ca1c7de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:51:41.316275 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.316264 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-kube-api-access-g4r5t" (OuterVolumeSpecName: "kube-api-access-g4r5t") pod "e85f1ce5-0da7-4760-935e-bdb60ca1c7de" (UID: "e85f1ce5-0da7-4760-935e-bdb60ca1c7de"). InnerVolumeSpecName "kube-api-access-g4r5t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:51:41.415783 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.415753 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:51:41.415783 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.415780 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4r5t\" (UniqueName: \"kubernetes.io/projected/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-kube-api-access-g4r5t\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:51:41.415783 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.415789 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:51:41.415988 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.415798 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e85f1ce5-0da7-4760-935e-bdb60ca1c7de-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:51:41.780188 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.780153 2583 generic.go:356] "Generic (PLEG): container finished" podID="e85f1ce5-0da7-4760-935e-bdb60ca1c7de" containerID="d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b" exitCode=0 Sep 05 06:51:41.780370 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.780208 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" Sep 05 06:51:41.780370 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.780244 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" event={"ID":"e85f1ce5-0da7-4760-935e-bdb60ca1c7de","Type":"ContainerDied","Data":"d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b"} Sep 05 06:51:41.780370 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.780309 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb" event={"ID":"e85f1ce5-0da7-4760-935e-bdb60ca1c7de","Type":"ContainerDied","Data":"c7417aafe844cd05c292db081891e6833e6a103cb03bac540ab91f552d37eb7a"} Sep 05 06:51:41.780370 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.780336 2583 scope.go:117] "RemoveContainer" containerID="d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b" Sep 05 06:51:41.794313 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.794291 2583 scope.go:117] "RemoveContainer" containerID="d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b" Sep 05 06:51:41.794651 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:51:41.794631 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b\": container with ID starting with d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b not found: ID does not exist" containerID="d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b" Sep 05 06:51:41.794742 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.794662 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b"} err="failed to get container status \"d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b\": rpc error: code = NotFound desc = could not find container \"d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b\": container with ID starting with d6305723332e71b2e92ab3bbde0a37f69b49884577e7150baa43070735aca69b not found: ID does not exist" Sep 05 06:51:41.803956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.803934 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb"] Sep 05 06:51:41.809345 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.809324 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73c192-bf5844755-7c5cb"] Sep 05 06:51:41.823294 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:51:41.823269 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e85f1ce5-0da7-4760-935e-bdb60ca1c7de" path="/var/lib/kubelet/pods/e85f1ce5-0da7-4760-935e-bdb60ca1c7de/volumes" Sep 05 06:52:04.141043 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.140999 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m"] Sep 05 06:52:04.141443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.141275 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" podUID="87306c74-150b-4177-b2f3-f1421cc7b56e" containerName="maas-api" containerID="cri-o://68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c" gracePeriod=30 Sep 05 06:52:04.408701 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.408675 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:52:04.541172 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.541127 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-maas-api-tls\") pod \"87306c74-150b-4177-b2f3-f1421cc7b56e\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " Sep 05 06:52:04.541351 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.541244 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-metrics-certs\") pod \"87306c74-150b-4177-b2f3-f1421cc7b56e\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " Sep 05 06:52:04.541463 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.541392 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87306c74-150b-4177-b2f3-f1421cc7b56e-openshift-service-ca\") pod \"87306c74-150b-4177-b2f3-f1421cc7b56e\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " Sep 05 06:52:04.541463 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.541455 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8skcr\" (UniqueName: \"kubernetes.io/projected/87306c74-150b-4177-b2f3-f1421cc7b56e-kube-api-access-8skcr\") pod \"87306c74-150b-4177-b2f3-f1421cc7b56e\" (UID: \"87306c74-150b-4177-b2f3-f1421cc7b56e\") " Sep 05 06:52:04.542304 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.542266 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87306c74-150b-4177-b2f3-f1421cc7b56e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87306c74-150b-4177-b2f3-f1421cc7b56e" (UID: "87306c74-150b-4177-b2f3-f1421cc7b56e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:52:04.544315 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.544286 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "87306c74-150b-4177-b2f3-f1421cc7b56e" (UID: "87306c74-150b-4177-b2f3-f1421cc7b56e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:52:04.544771 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.544746 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87306c74-150b-4177-b2f3-f1421cc7b56e" (UID: "87306c74-150b-4177-b2f3-f1421cc7b56e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:52:04.544771 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.544764 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87306c74-150b-4177-b2f3-f1421cc7b56e-kube-api-access-8skcr" (OuterVolumeSpecName: "kube-api-access-8skcr") pod "87306c74-150b-4177-b2f3-f1421cc7b56e" (UID: "87306c74-150b-4177-b2f3-f1421cc7b56e"). InnerVolumeSpecName "kube-api-access-8skcr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:52:04.643377 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.643285 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:04.643377 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.643318 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87306c74-150b-4177-b2f3-f1421cc7b56e-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:04.643377 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.643327 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8skcr\" (UniqueName: \"kubernetes.io/projected/87306c74-150b-4177-b2f3-f1421cc7b56e-kube-api-access-8skcr\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:04.643377 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.643337 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87306c74-150b-4177-b2f3-f1421cc7b56e-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:04.921526 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.921419 2583 generic.go:356] "Generic (PLEG): container finished" podID="87306c74-150b-4177-b2f3-f1421cc7b56e" containerID="68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c" exitCode=0 Sep 05 06:52:04.921526 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.921463 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" Sep 05 06:52:04.921526 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.921497 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" event={"ID":"87306c74-150b-4177-b2f3-f1421cc7b56e","Type":"ContainerDied","Data":"68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c"} Sep 05 06:52:04.921781 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.921544 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m" event={"ID":"87306c74-150b-4177-b2f3-f1421cc7b56e","Type":"ContainerDied","Data":"a410e7a16e0bfe67b8a98d60c3ccf6a13a95b04dc18664ab4ca5b9fe9e7c4661"} Sep 05 06:52:04.921781 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.921566 2583 scope.go:117] "RemoveContainer" containerID="68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c" Sep 05 06:52:04.934337 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.934319 2583 scope.go:117] "RemoveContainer" containerID="68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c" Sep 05 06:52:04.934623 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:52:04.934604 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c\": container with ID starting with 68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c not found: ID does not exist" containerID="68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c" Sep 05 06:52:04.934682 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.934632 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c"} err="failed to get container status \"68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c\": rpc error: code = NotFound desc = could not find container \"68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c\": container with ID starting with 68df31176effae88019dbc74e9e0a28eac32918d2261195623b28c83f323608c not found: ID does not exist" Sep 05 06:52:04.946608 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.946582 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m"] Sep 05 06:52:04.950894 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:04.950873 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d91c81-6c879bd685-ngb2m"] Sep 05 06:52:05.822858 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:05.822825 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87306c74-150b-4177-b2f3-f1421cc7b56e" path="/var/lib/kubelet/pods/87306c74-150b-4177-b2f3-f1421cc7b56e/volumes" Sep 05 06:52:21.668273 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.668234 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg"] Sep 05 06:52:21.668806 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.668789 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e85f1ce5-0da7-4760-935e-bdb60ca1c7de" containerName="maas-api" Sep 05 06:52:21.668851 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.668807 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e85f1ce5-0da7-4760-935e-bdb60ca1c7de" containerName="maas-api" Sep 05 06:52:21.668851 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.668822 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87306c74-150b-4177-b2f3-f1421cc7b56e" containerName="maas-api" Sep 05 06:52:21.668851 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.668827 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="87306c74-150b-4177-b2f3-f1421cc7b56e" containerName="maas-api" Sep 05 06:52:21.668944 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.668911 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e85f1ce5-0da7-4760-935e-bdb60ca1c7de" containerName="maas-api" Sep 05 06:52:21.668944 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.668922 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="87306c74-150b-4177-b2f3-f1421cc7b56e" containerName="maas-api" Sep 05 06:52:21.673700 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.673673 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.676015 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.675992 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-73a747-serving-cert\"" Sep 05 06:52:21.685048 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.685024 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg"] Sep 05 06:52:21.804301 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.804257 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whjnz\" (UniqueName: \"kubernetes.io/projected/4f2e6b32-06cb-4bec-b873-da839e7dd49b-kube-api-access-whjnz\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.804532 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.804404 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-metrics-certs\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.804532 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.804443 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.804532 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.804524 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e6b32-06cb-4bec-b873-da839e7dd49b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.905413 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.905374 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whjnz\" (UniqueName: \"kubernetes.io/projected/4f2e6b32-06cb-4bec-b873-da839e7dd49b-kube-api-access-whjnz\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.905618 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.905445 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-metrics-certs\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.905686 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.905614 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.905746 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.905695 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e6b32-06cb-4bec-b873-da839e7dd49b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.905746 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:52:21.905702 2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-serving-cert: secret "maas-api-e2e-ipp-a-73a747-serving-cert" not found Sep 05 06:52:21.905856 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:52:21.905778 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-maas-api-tls podName:4f2e6b32-06cb-4bec-b873-da839e7dd49b nodeName:}" failed. No retries permitted until 2026-09-05 06:52:22.405756679 +0000 UTC m=+1639.089261576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-maas-api-tls") pod "maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" (UID: "4f2e6b32-06cb-4bec-b873-da839e7dd49b") : secret "maas-api-e2e-ipp-a-73a747-serving-cert" not found Sep 05 06:52:21.906309 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.906289 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e6b32-06cb-4bec-b873-da839e7dd49b-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.908065 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.908044 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-metrics-certs\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:21.913360 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:21.913336 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whjnz\" (UniqueName: \"kubernetes.io/projected/4f2e6b32-06cb-4bec-b873-da839e7dd49b-kube-api-access-whjnz\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:22.409430 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.409394 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:22.411812 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.411784 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-maas-api-tls\") pod \"maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:22.486234 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.486197 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4"] Sep 05 06:52:22.486540 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.486461 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" podUID="7d788851-50b3-40f1-8299-de8adde55bb0" containerName="maas-api" containerID="cri-o://829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a" gracePeriod=30 Sep 05 06:52:22.590692 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.590661 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:22.768035 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.768006 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:52:22.814808 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.814773 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89fsc\" (UniqueName: \"kubernetes.io/projected/7d788851-50b3-40f1-8299-de8adde55bb0-kube-api-access-89fsc\") pod \"7d788851-50b3-40f1-8299-de8adde55bb0\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " Sep 05 06:52:22.814808 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.814814 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-metrics-certs\") pod \"7d788851-50b3-40f1-8299-de8adde55bb0\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " Sep 05 06:52:22.815058 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.814850 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-maas-api-tls\") pod \"7d788851-50b3-40f1-8299-de8adde55bb0\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " Sep 05 06:52:22.815058 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.815026 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d788851-50b3-40f1-8299-de8adde55bb0-openshift-service-ca\") pod \"7d788851-50b3-40f1-8299-de8adde55bb0\" (UID: \"7d788851-50b3-40f1-8299-de8adde55bb0\") " Sep 05 06:52:22.815634 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.815604 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d788851-50b3-40f1-8299-de8adde55bb0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d788851-50b3-40f1-8299-de8adde55bb0" (UID: "7d788851-50b3-40f1-8299-de8adde55bb0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:52:22.817749 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.817526 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d788851-50b3-40f1-8299-de8adde55bb0" (UID: "7d788851-50b3-40f1-8299-de8adde55bb0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:52:22.817749 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.817734 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7d788851-50b3-40f1-8299-de8adde55bb0" (UID: "7d788851-50b3-40f1-8299-de8adde55bb0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:52:22.817921 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.817880 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d788851-50b3-40f1-8299-de8adde55bb0-kube-api-access-89fsc" (OuterVolumeSpecName: "kube-api-access-89fsc") pod "7d788851-50b3-40f1-8299-de8adde55bb0" (UID: "7d788851-50b3-40f1-8299-de8adde55bb0"). InnerVolumeSpecName "kube-api-access-89fsc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:52:22.876263 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.876229 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg"] Sep 05 06:52:22.879663 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:52:22.879630 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2e6b32_06cb_4bec_b873_da839e7dd49b.slice/crio-68d75201dc5168b7459ab3a77f0c0c2b42ee39cedd4605f834e6f2a328bdcc18 WatchSource:0}: Error finding container 68d75201dc5168b7459ab3a77f0c0c2b42ee39cedd4605f834e6f2a328bdcc18: Status 404 returned error can't find the container with id 68d75201dc5168b7459ab3a77f0c0c2b42ee39cedd4605f834e6f2a328bdcc18 Sep 05 06:52:22.916618 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.916541 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89fsc\" (UniqueName: \"kubernetes.io/projected/7d788851-50b3-40f1-8299-de8adde55bb0-kube-api-access-89fsc\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:22.916618 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.916574 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:22.916618 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.916592 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d788851-50b3-40f1-8299-de8adde55bb0-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:22.916618 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:22.916608 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d788851-50b3-40f1-8299-de8adde55bb0-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:23.032964 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.032929 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" event={"ID":"4f2e6b32-06cb-4bec-b873-da839e7dd49b","Type":"ContainerStarted","Data":"68d75201dc5168b7459ab3a77f0c0c2b42ee39cedd4605f834e6f2a328bdcc18"} Sep 05 06:52:23.034304 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.034278 2583 generic.go:356] "Generic (PLEG): container finished" podID="7d788851-50b3-40f1-8299-de8adde55bb0" containerID="829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a" exitCode=0 Sep 05 06:52:23.034428 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.034330 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" Sep 05 06:52:23.034428 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.034371 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" event={"ID":"7d788851-50b3-40f1-8299-de8adde55bb0","Type":"ContainerDied","Data":"829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a"} Sep 05 06:52:23.034526 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.034427 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4" event={"ID":"7d788851-50b3-40f1-8299-de8adde55bb0","Type":"ContainerDied","Data":"28dcc836ca39bd6108814932f3089a7c0ff212d2a61db2f09f318b42999abaa6"} Sep 05 06:52:23.034526 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.034451 2583 scope.go:117] "RemoveContainer" containerID="829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a" Sep 05 06:52:23.044810 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.044794 2583 scope.go:117] "RemoveContainer" containerID="829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a" Sep 05 06:52:23.045016 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:52:23.044997 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a\": container with ID starting with 829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a not found: ID does not exist" containerID="829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a" Sep 05 06:52:23.045073 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.045023 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a"} err="failed to get container status \"829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a\": rpc error: code = NotFound desc = could not find container \"829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a\": container with ID starting with 829848c8f8c5671d733b3eb235f0a9f262ba17810d4ddcbea2c2ebe60413e20a not found: ID does not exist" Sep 05 06:52:23.058420 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.058395 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4"] Sep 05 06:52:23.061757 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.061735 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-eac764-59d8fdc679-8x2r4"] Sep 05 06:52:23.828065 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:23.828032 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d788851-50b3-40f1-8299-de8adde55bb0" path="/var/lib/kubelet/pods/7d788851-50b3-40f1-8299-de8adde55bb0/volumes" Sep 05 06:52:24.042004 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:24.041964 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" event={"ID":"4f2e6b32-06cb-4bec-b873-da839e7dd49b","Type":"ContainerStarted","Data":"4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c"} Sep 05 06:52:24.042201 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:24.042022 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:24.060537 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:24.060488 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" podStartSLOduration=2.683993817 podStartE2EDuration="3.060457824s" totalImagesPullingTime="376.464007ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:52:24.058625642 +0000 UTC m=+1640.742130563" watchObservedRunningTime="2026-09-05 06:52:24.060457824 +0000 UTC m=+1640.743962731" Sep 05 06:52:30.054399 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:30.054367 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:52:40.646170 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:40.646131 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp"] Sep 05 06:52:40.646673 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:40.646445 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" podUID="9748a856-4932-4830-9e9b-9564e9402e45" containerName="maas-api" containerID="cri-o://aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab" gracePeriod=30 Sep 05 06:52:40.908114 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:40.908051 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:52:41.004849 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.004821 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-metrics-certs\") pod \"9748a856-4932-4830-9e9b-9564e9402e45\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " Sep 05 06:52:41.005059 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.004885 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-maas-api-tls\") pod \"9748a856-4932-4830-9e9b-9564e9402e45\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " Sep 05 06:52:41.005059 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.004915 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fws5t\" (UniqueName: \"kubernetes.io/projected/9748a856-4932-4830-9e9b-9564e9402e45-kube-api-access-fws5t\") pod \"9748a856-4932-4830-9e9b-9564e9402e45\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " Sep 05 06:52:41.005059 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.004933 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9748a856-4932-4830-9e9b-9564e9402e45-openshift-service-ca\") pod \"9748a856-4932-4830-9e9b-9564e9402e45\" (UID: \"9748a856-4932-4830-9e9b-9564e9402e45\") " Sep 05 06:52:41.005445 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.005403 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9748a856-4932-4830-9e9b-9564e9402e45-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9748a856-4932-4830-9e9b-9564e9402e45" (UID: "9748a856-4932-4830-9e9b-9564e9402e45"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:52:41.006880 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.006852 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9748a856-4932-4830-9e9b-9564e9402e45" (UID: "9748a856-4932-4830-9e9b-9564e9402e45"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:52:41.007452 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.007418 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9748a856-4932-4830-9e9b-9564e9402e45-kube-api-access-fws5t" (OuterVolumeSpecName: "kube-api-access-fws5t") pod "9748a856-4932-4830-9e9b-9564e9402e45" (UID: "9748a856-4932-4830-9e9b-9564e9402e45"). InnerVolumeSpecName "kube-api-access-fws5t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:52:41.007452 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.007438 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9748a856-4932-4830-9e9b-9564e9402e45" (UID: "9748a856-4932-4830-9e9b-9564e9402e45"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:52:41.105892 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.105861 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fws5t\" (UniqueName: \"kubernetes.io/projected/9748a856-4932-4830-9e9b-9564e9402e45-kube-api-access-fws5t\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:41.105892 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.105892 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9748a856-4932-4830-9e9b-9564e9402e45-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:41.106087 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.105906 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:41.106087 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.105918 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9748a856-4932-4830-9e9b-9564e9402e45-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:52:41.141731 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.141702 2583 generic.go:356] "Generic (PLEG): container finished" podID="9748a856-4932-4830-9e9b-9564e9402e45" containerID="aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab" exitCode=0 Sep 05 06:52:41.141889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.141748 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" event={"ID":"9748a856-4932-4830-9e9b-9564e9402e45","Type":"ContainerDied","Data":"aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab"} Sep 05 06:52:41.141889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.141754 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" Sep 05 06:52:41.141889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.141790 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp" event={"ID":"9748a856-4932-4830-9e9b-9564e9402e45","Type":"ContainerDied","Data":"dce45c8c6dc5fc2e7eaf2e7065c76dbef706759e707c5152d3be39c2bb2a411a"} Sep 05 06:52:41.141889 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.141812 2583 scope.go:117] "RemoveContainer" containerID="aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab" Sep 05 06:52:41.152419 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.152399 2583 scope.go:117] "RemoveContainer" containerID="aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab" Sep 05 06:52:41.152699 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:52:41.152680 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab\": container with ID starting with aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab not found: ID does not exist" containerID="aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab" Sep 05 06:52:41.152763 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.152708 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab"} err="failed to get container status \"aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab\": rpc error: code = NotFound desc = could not find container \"aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab\": container with ID starting with aeda0018ce64d6a4815c75c58d48817eef23eae35050f5d7fe99feeaa4d329ab not found: ID does not exist" Sep 05 06:52:41.170423 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.170356 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp"] Sep 05 06:52:41.176884 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.176844 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-32e859-997b74dcb-z7nvp"] Sep 05 06:52:41.825627 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:41.825596 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9748a856-4932-4830-9e9b-9564e9402e45" path="/var/lib/kubelet/pods/9748a856-4932-4830-9e9b-9564e9402e45/volumes" Sep 05 06:52:51.055755 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.055708 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff"] Sep 05 06:52:51.056248 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.056233 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d788851-50b3-40f1-8299-de8adde55bb0" containerName="maas-api" Sep 05 06:52:51.056298 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.056249 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d788851-50b3-40f1-8299-de8adde55bb0" containerName="maas-api" Sep 05 06:52:51.056298 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.056258 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9748a856-4932-4830-9e9b-9564e9402e45" containerName="maas-api" Sep 05 06:52:51.056298 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.056264 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9748a856-4932-4830-9e9b-9564e9402e45" containerName="maas-api" Sep 05 06:52:51.056390 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.056343 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9748a856-4932-4830-9e9b-9564e9402e45" containerName="maas-api" Sep 05 06:52:51.056390 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.056352 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d788851-50b3-40f1-8299-de8adde55bb0" containerName="maas-api" Sep 05 06:52:51.060894 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.060875 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.064971 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.064947 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-930288-serving-cert\"" Sep 05 06:52:51.076099 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.076076 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff"] Sep 05 06:52:51.211345 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.211303 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfrnh\" (UniqueName: \"kubernetes.io/projected/157cf57d-0286-4fbf-80a4-106f2be82efd-kube-api-access-qfrnh\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.211345 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.211346 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-metrics-certs\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.211644 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.211438 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.211644 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.211501 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/157cf57d-0286-4fbf-80a4-106f2be82efd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.312849 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.312754 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfrnh\" (UniqueName: \"kubernetes.io/projected/157cf57d-0286-4fbf-80a4-106f2be82efd-kube-api-access-qfrnh\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.312849 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.312793 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-metrics-certs\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.313205 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.312926 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.313205 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.312983 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/157cf57d-0286-4fbf-80a4-106f2be82efd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.313763 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.313735 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/157cf57d-0286-4fbf-80a4-106f2be82efd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.315372 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.315349 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.315469 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.315456 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-metrics-certs\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.324010 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.323983 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfrnh\" (UniqueName: \"kubernetes.io/projected/157cf57d-0286-4fbf-80a4-106f2be82efd-kube-api-access-qfrnh\") pod \"maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.372392 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.372346 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:51.652243 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:51.652212 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff"] Sep 05 06:52:51.653901 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:52:51.653878 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod157cf57d_0286_4fbf_80a4_106f2be82efd.slice/crio-fc33034886e3927c5fd8ba9ff9aae4117f93c1d462ed8180631c24ef8e85b39a WatchSource:0}: Error finding container fc33034886e3927c5fd8ba9ff9aae4117f93c1d462ed8180631c24ef8e85b39a: Status 404 returned error can't find the container with id fc33034886e3927c5fd8ba9ff9aae4117f93c1d462ed8180631c24ef8e85b39a Sep 05 06:52:52.209144 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:52.209114 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" event={"ID":"157cf57d-0286-4fbf-80a4-106f2be82efd","Type":"ContainerStarted","Data":"74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c"} Sep 05 06:52:52.209540 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:52.209165 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" event={"ID":"157cf57d-0286-4fbf-80a4-106f2be82efd","Type":"ContainerStarted","Data":"fc33034886e3927c5fd8ba9ff9aae4117f93c1d462ed8180631c24ef8e85b39a"} Sep 05 06:52:52.209540 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:52.209287 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:52:52.227376 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:52.227328 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" podStartSLOduration=0.875856855 podStartE2EDuration="1.227312165s" totalImagesPullingTime="351.45531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:52:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:52:52.225642976 +0000 UTC m=+1668.909147885" watchObservedRunningTime="2026-09-05 06:52:52.227312165 +0000 UTC m=+1668.910817071" Sep 05 06:52:58.220757 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:52:58.220728 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:54:03.422362 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.422325 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w"] Sep 05 06:54:03.426842 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.426820 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.429065 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.429039 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-703730-serving-cert\"" Sep 05 06:54:03.438274 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.438249 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w"] Sep 05 06:54:03.486001 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.485966 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40bab66e-0382-4d7c-ae4b-c870b4017c79-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.486191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.486012 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxfmn\" (UniqueName: \"kubernetes.io/projected/40bab66e-0382-4d7c-ae4b-c870b4017c79-kube-api-access-jxfmn\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.486191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.486171 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.486289 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.486227 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.587341 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.587307 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.587539 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.587364 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.587539 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.587391 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40bab66e-0382-4d7c-ae4b-c870b4017c79-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.587539 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.587414 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxfmn\" (UniqueName: \"kubernetes.io/projected/40bab66e-0382-4d7c-ae4b-c870b4017c79-kube-api-access-jxfmn\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.588015 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.587993 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40bab66e-0382-4d7c-ae4b-c870b4017c79-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.589862 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.589843 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.589923 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.589899 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.595698 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.595669 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxfmn\" (UniqueName: \"kubernetes.io/projected/40bab66e-0382-4d7c-ae4b-c870b4017c79-kube-api-access-jxfmn\") pod \"maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:03.740160 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:03.740127 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:04.017536 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:04.017502 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w"] Sep 05 06:54:04.019292 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:54:04.019263 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40bab66e_0382_4d7c_ae4b_c870b4017c79.slice/crio-347d7f03e27cd76a58bdc4c312e3e3f681ad7eddb92902c2336e60538d29f056 WatchSource:0}: Error finding container 347d7f03e27cd76a58bdc4c312e3e3f681ad7eddb92902c2336e60538d29f056: Status 404 returned error can't find the container with id 347d7f03e27cd76a58bdc4c312e3e3f681ad7eddb92902c2336e60538d29f056 Sep 05 06:54:04.652785 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:04.652748 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" event={"ID":"40bab66e-0382-4d7c-ae4b-c870b4017c79","Type":"ContainerStarted","Data":"652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88"} Sep 05 06:54:04.653180 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:04.652815 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" event={"ID":"40bab66e-0382-4d7c-ae4b-c870b4017c79","Type":"ContainerStarted","Data":"347d7f03e27cd76a58bdc4c312e3e3f681ad7eddb92902c2336e60538d29f056"} Sep 05 06:54:04.653180 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:04.652946 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:04.675388 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:04.675298 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" podStartSLOduration=1.277836589 podStartE2EDuration="1.675283876s" totalImagesPullingTime="397.447287ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:54:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:54:04.67254539 +0000 UTC m=+1741.356050298" watchObservedRunningTime="2026-09-05 06:54:04.675283876 +0000 UTC m=+1741.358788829" Sep 05 06:54:10.665788 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:10.665758 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:34.097986 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.097514 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w"] Sep 05 06:54:34.097986 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.097778 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" podUID="40bab66e-0382-4d7c-ae4b-c870b4017c79" containerName="maas-api" containerID="cri-o://652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88" gracePeriod=30 Sep 05 06:54:34.359777 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.359721 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:34.378530 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.378466 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-maas-api-tls\") pod \"40bab66e-0382-4d7c-ae4b-c870b4017c79\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " Sep 05 06:54:34.378682 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.378576 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-metrics-certs\") pod \"40bab66e-0382-4d7c-ae4b-c870b4017c79\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " Sep 05 06:54:34.379041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.378674 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxfmn\" (UniqueName: \"kubernetes.io/projected/40bab66e-0382-4d7c-ae4b-c870b4017c79-kube-api-access-jxfmn\") pod \"40bab66e-0382-4d7c-ae4b-c870b4017c79\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " Sep 05 06:54:34.380002 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.379088 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40bab66e-0382-4d7c-ae4b-c870b4017c79-openshift-service-ca\") pod \"40bab66e-0382-4d7c-ae4b-c870b4017c79\" (UID: \"40bab66e-0382-4d7c-ae4b-c870b4017c79\") " Sep 05 06:54:34.380002 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.379834 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40bab66e-0382-4d7c-ae4b-c870b4017c79-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40bab66e-0382-4d7c-ae4b-c870b4017c79" (UID: "40bab66e-0382-4d7c-ae4b-c870b4017c79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:54:34.381218 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.381144 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40bab66e-0382-4d7c-ae4b-c870b4017c79" (UID: "40bab66e-0382-4d7c-ae4b-c870b4017c79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:54:34.381849 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.381817 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "40bab66e-0382-4d7c-ae4b-c870b4017c79" (UID: "40bab66e-0382-4d7c-ae4b-c870b4017c79"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:54:34.382231 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.382201 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40bab66e-0382-4d7c-ae4b-c870b4017c79-kube-api-access-jxfmn" (OuterVolumeSpecName: "kube-api-access-jxfmn") pod "40bab66e-0382-4d7c-ae4b-c870b4017c79" (UID: "40bab66e-0382-4d7c-ae4b-c870b4017c79"). InnerVolumeSpecName "kube-api-access-jxfmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:54:34.480561 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.480525 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40bab66e-0382-4d7c-ae4b-c870b4017c79-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:54:34.480561 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.480558 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:54:34.480770 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.480576 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bab66e-0382-4d7c-ae4b-c870b4017c79-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:54:34.480770 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.480585 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxfmn\" (UniqueName: \"kubernetes.io/projected/40bab66e-0382-4d7c-ae4b-c870b4017c79-kube-api-access-jxfmn\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:54:34.832283 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.832252 2583 generic.go:356] "Generic (PLEG): container finished" podID="40bab66e-0382-4d7c-ae4b-c870b4017c79" containerID="652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88" exitCode=0 Sep 05 06:54:34.832464 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.832300 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" Sep 05 06:54:34.832464 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.832350 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" event={"ID":"40bab66e-0382-4d7c-ae4b-c870b4017c79","Type":"ContainerDied","Data":"652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88"} Sep 05 06:54:34.832464 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.832398 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w" event={"ID":"40bab66e-0382-4d7c-ae4b-c870b4017c79","Type":"ContainerDied","Data":"347d7f03e27cd76a58bdc4c312e3e3f681ad7eddb92902c2336e60538d29f056"} Sep 05 06:54:34.832464 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.832421 2583 scope.go:117] "RemoveContainer" containerID="652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88" Sep 05 06:54:34.842434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.842414 2583 scope.go:117] "RemoveContainer" containerID="652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88" Sep 05 06:54:34.842717 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:54:34.842699 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88\": container with ID starting with 652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88 not found: ID does not exist" containerID="652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88" Sep 05 06:54:34.842793 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.842723 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88"} err="failed to get container status \"652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88\": rpc error: code = NotFound desc = could not find container \"652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88\": container with ID starting with 652ebdcfb9415a8b0d786ccdbd1b036d06d86b9a65ba47d4d34c306dbe80fe88 not found: ID does not exist" Sep 05 06:54:34.855507 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.855485 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w"] Sep 05 06:54:34.861387 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:34.861366 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-703730-847d9fcf94-zcd8w"] Sep 05 06:54:35.823261 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:54:35.823226 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40bab66e-0382-4d7c-ae4b-c870b4017c79" path="/var/lib/kubelet/pods/40bab66e-0382-4d7c-ae4b-c870b4017c79/volumes" Sep 05 06:55:00.237061 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.236977 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg"] Sep 05 06:55:00.237524 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.237227 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" podUID="4f2e6b32-06cb-4bec-b873-da839e7dd49b" containerName="maas-api" containerID="cri-o://4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c" gracePeriod=30 Sep 05 06:55:00.498446 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.498385 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:55:00.638689 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.638644 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-metrics-certs\") pod \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " Sep 05 06:55:00.638881 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.638725 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-maas-api-tls\") pod \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " Sep 05 06:55:00.638881 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.638753 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e6b32-06cb-4bec-b873-da839e7dd49b-openshift-service-ca\") pod \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " Sep 05 06:55:00.638881 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.638858 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whjnz\" (UniqueName: \"kubernetes.io/projected/4f2e6b32-06cb-4bec-b873-da839e7dd49b-kube-api-access-whjnz\") pod \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\" (UID: \"4f2e6b32-06cb-4bec-b873-da839e7dd49b\") " Sep 05 06:55:00.639171 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.639143 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2e6b32-06cb-4bec-b873-da839e7dd49b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f2e6b32-06cb-4bec-b873-da839e7dd49b" (UID: "4f2e6b32-06cb-4bec-b873-da839e7dd49b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:55:00.640787 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.640758 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4f2e6b32-06cb-4bec-b873-da839e7dd49b" (UID: "4f2e6b32-06cb-4bec-b873-da839e7dd49b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:55:00.640902 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.640790 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f2e6b32-06cb-4bec-b873-da839e7dd49b-kube-api-access-whjnz" (OuterVolumeSpecName: "kube-api-access-whjnz") pod "4f2e6b32-06cb-4bec-b873-da839e7dd49b" (UID: "4f2e6b32-06cb-4bec-b873-da839e7dd49b"). InnerVolumeSpecName "kube-api-access-whjnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:55:00.640902 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.640820 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f2e6b32-06cb-4bec-b873-da839e7dd49b" (UID: "4f2e6b32-06cb-4bec-b873-da839e7dd49b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:55:00.740189 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.740152 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2e6b32-06cb-4bec-b873-da839e7dd49b-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:55:00.740189 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.740183 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whjnz\" (UniqueName: \"kubernetes.io/projected/4f2e6b32-06cb-4bec-b873-da839e7dd49b-kube-api-access-whjnz\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:55:00.740189 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.740195 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:55:00.740407 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.740204 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f2e6b32-06cb-4bec-b873-da839e7dd49b-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:55:00.988179 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.988141 2583 generic.go:356] "Generic (PLEG): container finished" podID="4f2e6b32-06cb-4bec-b873-da839e7dd49b" containerID="4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c" exitCode=0 Sep 05 06:55:00.988339 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.988190 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" Sep 05 06:55:00.988339 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.988221 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" event={"ID":"4f2e6b32-06cb-4bec-b873-da839e7dd49b","Type":"ContainerDied","Data":"4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c"} Sep 05 06:55:00.988339 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.988274 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg" event={"ID":"4f2e6b32-06cb-4bec-b873-da839e7dd49b","Type":"ContainerDied","Data":"68d75201dc5168b7459ab3a77f0c0c2b42ee39cedd4605f834e6f2a328bdcc18"} Sep 05 06:55:00.988339 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.988296 2583 scope.go:117] "RemoveContainer" containerID="4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c" Sep 05 06:55:00.998400 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.998382 2583 scope.go:117] "RemoveContainer" containerID="4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c" Sep 05 06:55:00.998679 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:55:00.998660 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c\": container with ID starting with 4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c not found: ID does not exist" containerID="4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c" Sep 05 06:55:00.998740 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:00.998688 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c"} err="failed to get container status \"4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c\": rpc error: code = NotFound desc = could not find container \"4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c\": container with ID starting with 4c47c8752abb02a8584d88b1f63bde744a2e6fe8b59aea6d5e9ed43993fa619c not found: ID does not exist" Sep 05 06:55:01.011968 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:01.011944 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg"] Sep 05 06:55:01.014691 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:01.014669 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-73a747-78664d58f4-xfxqg"] Sep 05 06:55:01.823353 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:01.823318 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f2e6b32-06cb-4bec-b873-da839e7dd49b" path="/var/lib/kubelet/pods/4f2e6b32-06cb-4bec-b873-da839e7dd49b/volumes" Sep 05 06:55:03.966739 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:03.966709 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:55:03.974240 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:03.974213 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:55:03.974536 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:03.974517 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 06:55:03.981445 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:03.981420 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 06:55:59.866008 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:59.865969 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff"] Sep 05 06:55:59.866544 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:55:59.866229 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" podUID="157cf57d-0286-4fbf-80a4-106f2be82efd" containerName="maas-api" containerID="cri-o://74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c" gracePeriod=30 Sep 05 06:56:00.126678 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.126618 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:56:00.206612 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.206577 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-metrics-certs\") pod \"157cf57d-0286-4fbf-80a4-106f2be82efd\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " Sep 05 06:56:00.206800 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.206760 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfrnh\" (UniqueName: \"kubernetes.io/projected/157cf57d-0286-4fbf-80a4-106f2be82efd-kube-api-access-qfrnh\") pod \"157cf57d-0286-4fbf-80a4-106f2be82efd\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " Sep 05 06:56:00.206887 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.206872 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-maas-api-tls\") pod \"157cf57d-0286-4fbf-80a4-106f2be82efd\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " Sep 05 06:56:00.206972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.206956 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/157cf57d-0286-4fbf-80a4-106f2be82efd-openshift-service-ca\") pod \"157cf57d-0286-4fbf-80a4-106f2be82efd\" (UID: \"157cf57d-0286-4fbf-80a4-106f2be82efd\") " Sep 05 06:56:00.208012 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.207966 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/157cf57d-0286-4fbf-80a4-106f2be82efd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "157cf57d-0286-4fbf-80a4-106f2be82efd" (UID: "157cf57d-0286-4fbf-80a4-106f2be82efd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:56:00.209201 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.209177 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "157cf57d-0286-4fbf-80a4-106f2be82efd" (UID: "157cf57d-0286-4fbf-80a4-106f2be82efd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:56:00.209500 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.209461 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/157cf57d-0286-4fbf-80a4-106f2be82efd-kube-api-access-qfrnh" (OuterVolumeSpecName: "kube-api-access-qfrnh") pod "157cf57d-0286-4fbf-80a4-106f2be82efd" (UID: "157cf57d-0286-4fbf-80a4-106f2be82efd"). InnerVolumeSpecName "kube-api-access-qfrnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:56:00.210278 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.210249 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "157cf57d-0286-4fbf-80a4-106f2be82efd" (UID: "157cf57d-0286-4fbf-80a4-106f2be82efd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:56:00.308201 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.308158 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:00.308201 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.308193 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfrnh\" (UniqueName: \"kubernetes.io/projected/157cf57d-0286-4fbf-80a4-106f2be82efd-kube-api-access-qfrnh\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:00.308201 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.308203 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/157cf57d-0286-4fbf-80a4-106f2be82efd-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:00.308450 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.308212 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/157cf57d-0286-4fbf-80a4-106f2be82efd-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:00.349548 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.349512 2583 generic.go:356] "Generic (PLEG): container finished" podID="157cf57d-0286-4fbf-80a4-106f2be82efd" containerID="74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c" exitCode=0 Sep 05 06:56:00.349716 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.349562 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" Sep 05 06:56:00.349716 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.349599 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" event={"ID":"157cf57d-0286-4fbf-80a4-106f2be82efd","Type":"ContainerDied","Data":"74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c"} Sep 05 06:56:00.349716 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.349655 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff" event={"ID":"157cf57d-0286-4fbf-80a4-106f2be82efd","Type":"ContainerDied","Data":"fc33034886e3927c5fd8ba9ff9aae4117f93c1d462ed8180631c24ef8e85b39a"} Sep 05 06:56:00.349716 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.349678 2583 scope.go:117] "RemoveContainer" containerID="74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c" Sep 05 06:56:00.359963 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.359947 2583 scope.go:117] "RemoveContainer" containerID="74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c" Sep 05 06:56:00.360219 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:56:00.360201 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c\": container with ID starting with 74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c not found: ID does not exist" containerID="74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c" Sep 05 06:56:00.360270 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.360226 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c"} err="failed to get container status \"74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c\": rpc error: code = NotFound desc = could not find container \"74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c\": container with ID starting with 74cdaaa9ec52a17a9173a62a13f675a680eb3e61dd0314fd2a8427d1d8278f7c not found: ID does not exist" Sep 05 06:56:00.375672 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.375643 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff"] Sep 05 06:56:00.379028 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:00.378973 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-930288-8dd69fbdd-lhwff"] Sep 05 06:56:01.823656 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:01.823619 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="157cf57d-0286-4fbf-80a4-106f2be82efd" path="/var/lib/kubelet/pods/157cf57d-0286-4fbf-80a4-106f2be82efd/volumes" Sep 05 06:56:17.591027 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.590996 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5"] Sep 05 06:56:17.591456 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.591224 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" podUID="598a193a-d4ee-4295-92cc-6c65e4044830" containerName="maas-api" containerID="cri-o://6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56" gracePeriod=30 Sep 05 06:56:17.841139 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.841088 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:56:17.974077 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.974046 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxfz8\" (UniqueName: \"kubernetes.io/projected/598a193a-d4ee-4295-92cc-6c65e4044830-kube-api-access-pxfz8\") pod \"598a193a-d4ee-4295-92cc-6c65e4044830\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " Sep 05 06:56:17.974263 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.974101 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-metrics-certs\") pod \"598a193a-d4ee-4295-92cc-6c65e4044830\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " Sep 05 06:56:17.974263 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.974154 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/598a193a-d4ee-4295-92cc-6c65e4044830-openshift-service-ca\") pod \"598a193a-d4ee-4295-92cc-6c65e4044830\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " Sep 05 06:56:17.974263 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.974182 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-maas-api-tls\") pod \"598a193a-d4ee-4295-92cc-6c65e4044830\" (UID: \"598a193a-d4ee-4295-92cc-6c65e4044830\") " Sep 05 06:56:17.974585 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.974554 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/598a193a-d4ee-4295-92cc-6c65e4044830-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "598a193a-d4ee-4295-92cc-6c65e4044830" (UID: "598a193a-d4ee-4295-92cc-6c65e4044830"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:56:17.974910 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.974886 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/598a193a-d4ee-4295-92cc-6c65e4044830-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:17.976369 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.976346 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "598a193a-d4ee-4295-92cc-6c65e4044830" (UID: "598a193a-d4ee-4295-92cc-6c65e4044830"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:56:17.976369 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.976354 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/598a193a-d4ee-4295-92cc-6c65e4044830-kube-api-access-pxfz8" (OuterVolumeSpecName: "kube-api-access-pxfz8") pod "598a193a-d4ee-4295-92cc-6c65e4044830" (UID: "598a193a-d4ee-4295-92cc-6c65e4044830"). InnerVolumeSpecName "kube-api-access-pxfz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:56:17.976684 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:17.976666 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "598a193a-d4ee-4295-92cc-6c65e4044830" (UID: "598a193a-d4ee-4295-92cc-6c65e4044830"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:56:18.076284 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.076254 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:18.076284 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.076282 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxfz8\" (UniqueName: \"kubernetes.io/projected/598a193a-d4ee-4295-92cc-6c65e4044830-kube-api-access-pxfz8\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:18.076284 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.076292 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/598a193a-d4ee-4295-92cc-6c65e4044830-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:18.457452 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.457420 2583 generic.go:356] "Generic (PLEG): container finished" podID="598a193a-d4ee-4295-92cc-6c65e4044830" containerID="6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56" exitCode=0 Sep 05 06:56:18.457638 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.457462 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" event={"ID":"598a193a-d4ee-4295-92cc-6c65e4044830","Type":"ContainerDied","Data":"6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56"} Sep 05 06:56:18.457638 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.457470 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" Sep 05 06:56:18.457638 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.457521 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5" event={"ID":"598a193a-d4ee-4295-92cc-6c65e4044830","Type":"ContainerDied","Data":"3eb6394998a409877c2c8eeb68f82bd5bfe2e5bb181c805598ea05a08459a159"} Sep 05 06:56:18.457638 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.457547 2583 scope.go:117] "RemoveContainer" containerID="6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56" Sep 05 06:56:18.467090 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.467074 2583 scope.go:117] "RemoveContainer" containerID="6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56" Sep 05 06:56:18.467362 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:56:18.467343 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56\": container with ID starting with 6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56 not found: ID does not exist" containerID="6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56" Sep 05 06:56:18.467434 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.467369 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56"} err="failed to get container status \"6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56\": rpc error: code = NotFound desc = could not find container \"6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56\": container with ID starting with 6d44fbd6314d0619929fe4bc61a06c6dbcf2a70294c56c33a9f271d805949d56 not found: ID does not exist" Sep 05 06:56:18.479608 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.479589 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5"] Sep 05 06:56:18.483130 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:18.483108 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ef54fd-6566497dcd-cpxk5"] Sep 05 06:56:19.825604 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:19.825565 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="598a193a-d4ee-4295-92cc-6c65e4044830" path="/var/lib/kubelet/pods/598a193a-d4ee-4295-92cc-6c65e4044830/volumes" Sep 05 06:56:35.506468 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.506375 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8"] Sep 05 06:56:35.507058 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.506628 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" podUID="8089fd90-d284-4828-8d60-bd5f6d49065a" containerName="maas-api" containerID="cri-o://e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6" gracePeriod=30 Sep 05 06:56:35.765947 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.765887 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:56:35.837891 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.837859 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsz8s\" (UniqueName: \"kubernetes.io/projected/8089fd90-d284-4828-8d60-bd5f6d49065a-kube-api-access-dsz8s\") pod \"8089fd90-d284-4828-8d60-bd5f6d49065a\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " Sep 05 06:56:35.837891 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.837894 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8089fd90-d284-4828-8d60-bd5f6d49065a-openshift-service-ca\") pod \"8089fd90-d284-4828-8d60-bd5f6d49065a\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " Sep 05 06:56:35.838136 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.837939 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-maas-api-tls\") pod \"8089fd90-d284-4828-8d60-bd5f6d49065a\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " Sep 05 06:56:35.838136 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.837965 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-metrics-certs\") pod \"8089fd90-d284-4828-8d60-bd5f6d49065a\" (UID: \"8089fd90-d284-4828-8d60-bd5f6d49065a\") " Sep 05 06:56:35.838354 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.838327 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8089fd90-d284-4828-8d60-bd5f6d49065a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8089fd90-d284-4828-8d60-bd5f6d49065a" (UID: "8089fd90-d284-4828-8d60-bd5f6d49065a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:56:35.840134 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.840103 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8089fd90-d284-4828-8d60-bd5f6d49065a" (UID: "8089fd90-d284-4828-8d60-bd5f6d49065a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:56:35.840842 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.840715 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8089fd90-d284-4828-8d60-bd5f6d49065a-kube-api-access-dsz8s" (OuterVolumeSpecName: "kube-api-access-dsz8s") pod "8089fd90-d284-4828-8d60-bd5f6d49065a" (UID: "8089fd90-d284-4828-8d60-bd5f6d49065a"). InnerVolumeSpecName "kube-api-access-dsz8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:56:35.840842 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.840729 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8089fd90-d284-4828-8d60-bd5f6d49065a" (UID: "8089fd90-d284-4828-8d60-bd5f6d49065a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:56:35.939302 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.939268 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsz8s\" (UniqueName: \"kubernetes.io/projected/8089fd90-d284-4828-8d60-bd5f6d49065a-kube-api-access-dsz8s\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:35.939302 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.939295 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8089fd90-d284-4828-8d60-bd5f6d49065a-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:35.939302 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.939306 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:35.939549 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:35.939315 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8089fd90-d284-4828-8d60-bd5f6d49065a-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:36.567270 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:36.567235 2583 generic.go:356] "Generic (PLEG): container finished" podID="8089fd90-d284-4828-8d60-bd5f6d49065a" containerID="e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6" exitCode=0 Sep 05 06:56:36.567780 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:36.567278 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" Sep 05 06:56:36.567780 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:36.567327 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" event={"ID":"8089fd90-d284-4828-8d60-bd5f6d49065a","Type":"ContainerDied","Data":"e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6"} Sep 05 06:56:36.567780 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:36.567392 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8" event={"ID":"8089fd90-d284-4828-8d60-bd5f6d49065a","Type":"ContainerDied","Data":"b5d51132401cca5512b6b3fbaa111674e57c5cd1b0ff7fc321e9bcdeba04ead1"} Sep 05 06:56:36.567780 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:36.567430 2583 scope.go:117] "RemoveContainer" containerID="e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6" Sep 05 06:56:36.578443 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:36.578426 2583 scope.go:117] "RemoveContainer" containerID="e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6" Sep 05 06:56:36.578742 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:56:36.578723 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6\": container with ID starting with e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6 not found: ID does not exist" containerID="e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6" Sep 05 06:56:36.578798 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:36.578749 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6"} err="failed to get container status \"e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6\": rpc error: code = NotFound desc = could not find container \"e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6\": container with ID starting with e96fa80259161e7b798518fa9e74d5fca66126cd03d7b9c952d884175ce0c8f6 not found: ID does not exist" Sep 05 06:56:36.590655 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:36.590632 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8"] Sep 05 06:56:36.596457 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:36.596434 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-5b1d24-5557ffbbd5-5q7x8"] Sep 05 06:56:37.822999 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:37.822960 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8089fd90-d284-4828-8d60-bd5f6d49065a" path="/var/lib/kubelet/pods/8089fd90-d284-4828-8d60-bd5f6d49065a/volumes" Sep 05 06:56:42.233291 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:42.233257 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr"] Sep 05 06:56:42.233758 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:42.233528 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" containerID="cri-o://ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb" gracePeriod=30 Sep 05 06:56:44.829027 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:44.828987 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55889757f5-rdk4q"] Sep 05 06:56:44.829441 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:44.829218 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55889757f5-rdk4q" podUID="73644b49-45ee-441e-90c0-b99b829d5499" containerName="manager" containerID="cri-o://d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791" gracePeriod=30 Sep 05 06:56:45.070418 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.070394 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:56:45.124007 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.123925 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-webhook-cert\") pod \"73644b49-45ee-441e-90c0-b99b829d5499\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " Sep 05 06:56:45.124178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.124015 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwjq5\" (UniqueName: \"kubernetes.io/projected/73644b49-45ee-441e-90c0-b99b829d5499-kube-api-access-kwjq5\") pod \"73644b49-45ee-441e-90c0-b99b829d5499\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " Sep 05 06:56:45.124178 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.124099 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-metrics-certs\") pod \"73644b49-45ee-441e-90c0-b99b829d5499\" (UID: \"73644b49-45ee-441e-90c0-b99b829d5499\") " Sep 05 06:56:45.126034 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.125999 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "73644b49-45ee-441e-90c0-b99b829d5499" (UID: "73644b49-45ee-441e-90c0-b99b829d5499"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:56:45.126191 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.126164 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "73644b49-45ee-441e-90c0-b99b829d5499" (UID: "73644b49-45ee-441e-90c0-b99b829d5499"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:56:45.126429 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.126403 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73644b49-45ee-441e-90c0-b99b829d5499-kube-api-access-kwjq5" (OuterVolumeSpecName: "kube-api-access-kwjq5") pod "73644b49-45ee-441e-90c0-b99b829d5499" (UID: "73644b49-45ee-441e-90c0-b99b829d5499"). InnerVolumeSpecName "kube-api-access-kwjq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:56:45.225464 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.225431 2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-webhook-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:45.225464 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.225457 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwjq5\" (UniqueName: \"kubernetes.io/projected/73644b49-45ee-441e-90c0-b99b829d5499-kube-api-access-kwjq5\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:45.225464 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.225467 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73644b49-45ee-441e-90c0-b99b829d5499-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:45.620879 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.620844 2583 generic.go:356] "Generic (PLEG): container finished" podID="73644b49-45ee-441e-90c0-b99b829d5499" containerID="d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791" exitCode=0 Sep 05 06:56:45.621054 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.620888 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55889757f5-rdk4q" Sep 05 06:56:45.621054 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.620903 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55889757f5-rdk4q" event={"ID":"73644b49-45ee-441e-90c0-b99b829d5499","Type":"ContainerDied","Data":"d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791"} Sep 05 06:56:45.621054 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.620944 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55889757f5-rdk4q" event={"ID":"73644b49-45ee-441e-90c0-b99b829d5499","Type":"ContainerDied","Data":"b4c1f938724e13d9061924d374b4421e938b223970ddb69947dcdefd2d46f1b6"} Sep 05 06:56:45.621054 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.620969 2583 scope.go:117] "RemoveContainer" containerID="d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791" Sep 05 06:56:45.631753 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.631733 2583 scope.go:117] "RemoveContainer" containerID="d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791" Sep 05 06:56:45.631995 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:56:45.631975 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791\": container with ID starting with d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791 not found: ID does not exist" containerID="d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791" Sep 05 06:56:45.632067 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.632006 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791"} err="failed to get container status \"d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791\": rpc error: code = NotFound desc = could not find container \"d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791\": container with ID starting with d6117d92834bdeb16090e8a8a62da097cd7a0e57343c018ae1ec452018c90791 not found: ID does not exist" Sep 05 06:56:45.642956 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.642937 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55889757f5-rdk4q"] Sep 05 06:56:45.646041 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.646022 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55889757f5-rdk4q"] Sep 05 06:56:45.822946 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:45.822912 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73644b49-45ee-441e-90c0-b99b829d5499" path="/var/lib/kubelet/pods/73644b49-45ee-441e-90c0-b99b829d5499/volumes" Sep 05 06:56:47.485978 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.485951 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:56:47.546049 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.545983 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-podinfo\") pod \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " Sep 05 06:56:47.546049 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546036 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-credential-socket\") pod \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " Sep 05 06:56:47.546234 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546063 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-envoy\") pod \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " Sep 05 06:56:47.546234 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546093 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istiod-ca-cert\") pod \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " Sep 05 06:56:47.546234 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546120 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-socket\") pod \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " Sep 05 06:56:47.546234 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546140 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-data\") pod \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " Sep 05 06:56:47.546234 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546164 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-certs\") pod \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " Sep 05 06:56:47.546234 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546194 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-token\") pod \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " Sep 05 06:56:47.546621 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546260 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhjrk\" (UniqueName: \"kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-kube-api-access-qhjrk\") pod \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\" (UID: \"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff\") " Sep 05 06:56:47.546621 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546403 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" (UID: "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:56:47.546621 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546407 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" (UID: "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:56:47.546621 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546563 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" (UID: "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 06:56:47.546841 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546622 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" (UID: "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:56:47.546841 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546640 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-credential-socket\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:47.546841 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546661 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istiod-ca-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:47.546841 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546705 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-socket\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:47.546841 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.546748 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" (UID: "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:56:47.548193 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.548172 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" (UID: "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 05 06:56:47.548306 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.548291 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" (UID: "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 05 06:56:47.548727 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.548704 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" (UID: "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:56:47.548727 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.548718 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-kube-api-access-qhjrk" (OuterVolumeSpecName: "kube-api-access-qhjrk") pod "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" (UID: "7bb5ad39-4ffa-42d1-aca3-44839d5f52ff"). InnerVolumeSpecName "kube-api-access-qhjrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:56:47.637972 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.637941 2583 generic.go:356] "Generic (PLEG): container finished" podID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerID="ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb" exitCode=0 Sep 05 06:56:47.638084 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.637993 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" Sep 05 06:56:47.638084 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.638000 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" event={"ID":"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff","Type":"ContainerDied","Data":"ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb"} Sep 05 06:56:47.638084 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.638058 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr" event={"ID":"7bb5ad39-4ffa-42d1-aca3-44839d5f52ff","Type":"ContainerDied","Data":"70af58974953536e481660fe26b6a0dd1195323aa3580e52d7233c64360509de"} Sep 05 06:56:47.638196 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.638090 2583 scope.go:117] "RemoveContainer" containerID="ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb" Sep 05 06:56:47.647486 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.647452 2583 scope.go:117] "RemoveContainer" containerID="c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab" Sep 05 06:56:47.647609 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.647542 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-podinfo\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:47.647609 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.647560 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-envoy\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:47.647609 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.647568 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-data\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:47.647609 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.647584 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-workload-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:47.647609 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.647591 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-istio-token\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:47.647609 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.647600 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhjrk\" (UniqueName: \"kubernetes.io/projected/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff-kube-api-access-qhjrk\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:56:47.655959 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.655941 2583 scope.go:117] "RemoveContainer" containerID="ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb" Sep 05 06:56:47.656242 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:56:47.656221 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb\": container with ID starting with ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb not found: ID does not exist" containerID="ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb" Sep 05 06:56:47.656309 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.656253 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb"} err="failed to get container status \"ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb\": rpc error: code = NotFound desc = could not find container \"ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb\": container with ID starting with ce00f93b054cc5d7862514d7336319470e7ef16c5360bb97356bf494e73116fb not found: ID does not exist" Sep 05 06:56:47.656309 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.656275 2583 scope.go:117] "RemoveContainer" containerID="c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab" Sep 05 06:56:47.656528 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:56:47.656510 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab\": container with ID starting with c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab not found: ID does not exist" containerID="c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab" Sep 05 06:56:47.656590 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.656536 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab"} err="failed to get container status \"c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab\": rpc error: code = NotFound desc = could not find container \"c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab\": container with ID starting with c06104fb4d692a577d14a102dacc9ef4526dd78d96cf3c0a296cc927c9f8a7ab not found: ID does not exist" Sep 05 06:56:47.660540 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.660519 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr"] Sep 05 06:56:47.664511 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.664490 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5b1d24-openshift-default-66cdc54978-2rfzr"] Sep 05 06:56:47.822459 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:56:47.822379 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" path="/var/lib/kubelet/pods/7bb5ad39-4ffa-42d1-aca3-44839d5f52ff/volumes" Sep 05 06:57:46.486667 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.486620 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55889757f5-fbbz6"] Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487079 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2e6b32-06cb-4bec-b873-da839e7dd49b" containerName="maas-api" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487094 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2e6b32-06cb-4bec-b873-da839e7dd49b" containerName="maas-api" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487110 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="598a193a-d4ee-4295-92cc-6c65e4044830" containerName="maas-api" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487116 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="598a193a-d4ee-4295-92cc-6c65e4044830" containerName="maas-api" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487129 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487134 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487146 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73644b49-45ee-441e-90c0-b99b829d5499" containerName="manager" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487151 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="73644b49-45ee-441e-90c0-b99b829d5499" containerName="manager" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487157 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="157cf57d-0286-4fbf-80a4-106f2be82efd" containerName="maas-api" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487162 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="157cf57d-0286-4fbf-80a4-106f2be82efd" containerName="maas-api" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487170 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40bab66e-0382-4d7c-ae4b-c870b4017c79" containerName="maas-api" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487175 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="40bab66e-0382-4d7c-ae4b-c870b4017c79" containerName="maas-api" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487182 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8089fd90-d284-4828-8d60-bd5f6d49065a" containerName="maas-api" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487187 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8089fd90-d284-4828-8d60-bd5f6d49065a" containerName="maas-api" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487195 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" Sep 05 06:57:46.487220 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487200 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" Sep 05 06:57:46.487999 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487256 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" Sep 05 06:57:46.487999 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487267 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8089fd90-d284-4828-8d60-bd5f6d49065a" containerName="maas-api" Sep 05 06:57:46.487999 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487274 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="73644b49-45ee-441e-90c0-b99b829d5499" containerName="manager" Sep 05 06:57:46.487999 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487280 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="598a193a-d4ee-4295-92cc-6c65e4044830" containerName="maas-api" Sep 05 06:57:46.487999 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487286 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="40bab66e-0382-4d7c-ae4b-c870b4017c79" containerName="maas-api" Sep 05 06:57:46.487999 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487292 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2e6b32-06cb-4bec-b873-da839e7dd49b" containerName="maas-api" Sep 05 06:57:46.487999 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.487298 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="157cf57d-0286-4fbf-80a4-106f2be82efd" containerName="maas-api" Sep 05 06:57:46.490430 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.490415 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:46.493690 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.493664 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 05 06:57:46.493933 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.493916 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 05 06:57:46.494005 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.493949 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 05 06:57:46.494067 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.494009 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vlhj9\"" Sep 05 06:57:46.501804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.501781 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55889757f5-fbbz6"] Sep 05 06:57:46.667763 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.667729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jx9p\" (UniqueName: \"kubernetes.io/projected/73c75f33-484c-44ec-81b8-14d1abeecb11-kube-api-access-8jx9p\") pod \"maas-controller-55889757f5-fbbz6\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:46.667929 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.667793 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-metrics-certs\") pod \"maas-controller-55889757f5-fbbz6\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:46.667929 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.667893 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-webhook-cert\") pod \"maas-controller-55889757f5-fbbz6\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:46.769303 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.769214 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-webhook-cert\") pod \"maas-controller-55889757f5-fbbz6\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:46.769303 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.769261 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jx9p\" (UniqueName: \"kubernetes.io/projected/73c75f33-484c-44ec-81b8-14d1abeecb11-kube-api-access-8jx9p\") pod \"maas-controller-55889757f5-fbbz6\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:46.769562 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.769390 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-metrics-certs\") pod \"maas-controller-55889757f5-fbbz6\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:46.771745 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.771716 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-webhook-cert\") pod \"maas-controller-55889757f5-fbbz6\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:46.771858 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.771761 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-metrics-certs\") pod \"maas-controller-55889757f5-fbbz6\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:46.777831 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.777808 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jx9p\" (UniqueName: \"kubernetes.io/projected/73c75f33-484c-44ec-81b8-14d1abeecb11-kube-api-access-8jx9p\") pod \"maas-controller-55889757f5-fbbz6\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:46.802626 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:46.802604 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:47.079192 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:47.079169 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55889757f5-fbbz6"] Sep 05 06:57:47.081528 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:57:47.081495 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c75f33_484c_44ec_81b8_14d1abeecb11.slice/crio-85dd4c8bda1bac4c76e4e9a7bbf4d91cc5447d103e44c48c12021ceb691993ac WatchSource:0}: Error finding container 85dd4c8bda1bac4c76e4e9a7bbf4d91cc5447d103e44c48c12021ceb691993ac: Status 404 returned error can't find the container with id 85dd4c8bda1bac4c76e4e9a7bbf4d91cc5447d103e44c48c12021ceb691993ac Sep 05 06:57:47.082875 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:47.082859 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 05 06:57:47.974033 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:47.973952 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55889757f5-fbbz6" event={"ID":"73c75f33-484c-44ec-81b8-14d1abeecb11","Type":"ContainerStarted","Data":"5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150"} Sep 05 06:57:47.974033 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:47.974005 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55889757f5-fbbz6" event={"ID":"73c75f33-484c-44ec-81b8-14d1abeecb11","Type":"ContainerStarted","Data":"85dd4c8bda1bac4c76e4e9a7bbf4d91cc5447d103e44c48c12021ceb691993ac"} Sep 05 06:57:47.974412 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:47.974063 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:57:47.991535 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:47.991465 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55889757f5-fbbz6" podStartSLOduration=1.535653519 podStartE2EDuration="1.991448764s" totalImagesPullingTime="455.795245ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:57:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:57:47.989074925 +0000 UTC m=+1964.672579832" watchObservedRunningTime="2026-09-05 06:57:47.991448764 +0000 UTC m=+1964.674953672" Sep 05 06:57:58.981149 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:57:58.981068 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:58:08.420081 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.420039 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55889757f5-fbbz6"] Sep 05 06:58:08.420595 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.420363 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55889757f5-fbbz6" podUID="73c75f33-484c-44ec-81b8-14d1abeecb11" containerName="manager" containerID="cri-o://5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150" gracePeriod=30 Sep 05 06:58:08.667778 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.667753 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:58:08.765946 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.765917 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jx9p\" (UniqueName: \"kubernetes.io/projected/73c75f33-484c-44ec-81b8-14d1abeecb11-kube-api-access-8jx9p\") pod \"73c75f33-484c-44ec-81b8-14d1abeecb11\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " Sep 05 06:58:08.766105 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.766024 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-webhook-cert\") pod \"73c75f33-484c-44ec-81b8-14d1abeecb11\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " Sep 05 06:58:08.766105 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.766066 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-metrics-certs\") pod \"73c75f33-484c-44ec-81b8-14d1abeecb11\" (UID: \"73c75f33-484c-44ec-81b8-14d1abeecb11\") " Sep 05 06:58:08.768021 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.767984 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "73c75f33-484c-44ec-81b8-14d1abeecb11" (UID: "73c75f33-484c-44ec-81b8-14d1abeecb11"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:58:08.768132 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.768012 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "73c75f33-484c-44ec-81b8-14d1abeecb11" (UID: "73c75f33-484c-44ec-81b8-14d1abeecb11"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 06:58:08.768132 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.768064 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c75f33-484c-44ec-81b8-14d1abeecb11-kube-api-access-8jx9p" (OuterVolumeSpecName: "kube-api-access-8jx9p") pod "73c75f33-484c-44ec-81b8-14d1abeecb11" (UID: "73c75f33-484c-44ec-81b8-14d1abeecb11"). InnerVolumeSpecName "kube-api-access-8jx9p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 06:58:08.867308 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.867190 2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-webhook-cert\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:58:08.867308 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.867221 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73c75f33-484c-44ec-81b8-14d1abeecb11-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:58:08.867308 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:08.867239 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jx9p\" (UniqueName: \"kubernetes.io/projected/73c75f33-484c-44ec-81b8-14d1abeecb11-kube-api-access-8jx9p\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 06:58:09.091989 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.091901 2583 generic.go:356] "Generic (PLEG): container finished" podID="73c75f33-484c-44ec-81b8-14d1abeecb11" containerID="5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150" exitCode=0 Sep 05 06:58:09.091989 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.091942 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55889757f5-fbbz6" event={"ID":"73c75f33-484c-44ec-81b8-14d1abeecb11","Type":"ContainerDied","Data":"5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150"} Sep 05 06:58:09.091989 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.091948 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55889757f5-fbbz6" Sep 05 06:58:09.091989 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.091985 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55889757f5-fbbz6" event={"ID":"73c75f33-484c-44ec-81b8-14d1abeecb11","Type":"ContainerDied","Data":"85dd4c8bda1bac4c76e4e9a7bbf4d91cc5447d103e44c48c12021ceb691993ac"} Sep 05 06:58:09.092259 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.092008 2583 scope.go:117] "RemoveContainer" containerID="5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150" Sep 05 06:58:09.101768 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.101749 2583 scope.go:117] "RemoveContainer" containerID="5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150" Sep 05 06:58:09.102051 ip-10-0-142-24 kubenswrapper[2583]: E0905 06:58:09.102033 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150\": container with ID starting with 5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150 not found: ID does not exist" containerID="5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150" Sep 05 06:58:09.102097 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.102060 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150"} err="failed to get container status \"5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150\": rpc error: code = NotFound desc = could not find container \"5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150\": container with ID starting with 5907bb3672dce3ab2f8fc25889b567894f5a53351eee0c7afc37ec7778e01150 not found: ID does not exist" Sep 05 06:58:09.120147 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.120116 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55889757f5-fbbz6"] Sep 05 06:58:09.122388 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.122363 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55889757f5-fbbz6"] Sep 05 06:58:09.823285 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.823242 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73c75f33-484c-44ec-81b8-14d1abeecb11" path="/var/lib/kubelet/pods/73c75f33-484c-44ec-81b8-14d1abeecb11/volumes" Sep 05 06:58:09.969012 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.968976 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55889757f5-pxx66"] Sep 05 06:58:09.969405 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.969390 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c75f33-484c-44ec-81b8-14d1abeecb11" containerName="manager" Sep 05 06:58:09.969451 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.969408 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c75f33-484c-44ec-81b8-14d1abeecb11" containerName="manager" Sep 05 06:58:09.969512 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.969507 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="73c75f33-484c-44ec-81b8-14d1abeecb11" containerName="manager" Sep 05 06:58:09.969551 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.969517 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bb5ad39-4ffa-42d1-aca3-44839d5f52ff" containerName="istio-proxy" Sep 05 06:58:09.974005 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.973989 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:09.977392 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.977364 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 05 06:58:09.977535 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.977405 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 05 06:58:09.977535 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.977424 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 05 06:58:09.977535 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.977441 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vlhj9\"" Sep 05 06:58:09.981923 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:09.981901 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55889757f5-pxx66"] Sep 05 06:58:10.076151 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.076076 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vbgb\" (UniqueName: \"kubernetes.io/projected/0737305e-ae0a-4c9e-aea1-44062c2c2980-kube-api-access-2vbgb\") pod \"maas-controller-55889757f5-pxx66\" (UID: \"0737305e-ae0a-4c9e-aea1-44062c2c2980\") " pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:10.076151 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.076119 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0737305e-ae0a-4c9e-aea1-44062c2c2980-webhook-cert\") pod \"maas-controller-55889757f5-pxx66\" (UID: \"0737305e-ae0a-4c9e-aea1-44062c2c2980\") " pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:10.076315 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.076195 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0737305e-ae0a-4c9e-aea1-44062c2c2980-metrics-certs\") pod \"maas-controller-55889757f5-pxx66\" (UID: \"0737305e-ae0a-4c9e-aea1-44062c2c2980\") " pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:10.176640 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.176610 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0737305e-ae0a-4c9e-aea1-44062c2c2980-webhook-cert\") pod \"maas-controller-55889757f5-pxx66\" (UID: \"0737305e-ae0a-4c9e-aea1-44062c2c2980\") " pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:10.176804 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.176773 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0737305e-ae0a-4c9e-aea1-44062c2c2980-metrics-certs\") pod \"maas-controller-55889757f5-pxx66\" (UID: \"0737305e-ae0a-4c9e-aea1-44062c2c2980\") " pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:10.176888 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.176873 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vbgb\" (UniqueName: \"kubernetes.io/projected/0737305e-ae0a-4c9e-aea1-44062c2c2980-kube-api-access-2vbgb\") pod \"maas-controller-55889757f5-pxx66\" (UID: \"0737305e-ae0a-4c9e-aea1-44062c2c2980\") " pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:10.179011 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.178979 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0737305e-ae0a-4c9e-aea1-44062c2c2980-webhook-cert\") pod \"maas-controller-55889757f5-pxx66\" (UID: \"0737305e-ae0a-4c9e-aea1-44062c2c2980\") " pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:10.179133 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.179022 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0737305e-ae0a-4c9e-aea1-44062c2c2980-metrics-certs\") pod \"maas-controller-55889757f5-pxx66\" (UID: \"0737305e-ae0a-4c9e-aea1-44062c2c2980\") " pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:10.184846 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.184822 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vbgb\" (UniqueName: \"kubernetes.io/projected/0737305e-ae0a-4c9e-aea1-44062c2c2980-kube-api-access-2vbgb\") pod \"maas-controller-55889757f5-pxx66\" (UID: \"0737305e-ae0a-4c9e-aea1-44062c2c2980\") " pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:10.285418 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.285384 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:10.560467 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:10.560436 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55889757f5-pxx66"] Sep 05 06:58:10.561416 ip-10-0-142-24 kubenswrapper[2583]: W0905 06:58:10.561390 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0737305e_ae0a_4c9e_aea1_44062c2c2980.slice/crio-05a1839b624b0741419c0ca2ec69ac35af8228cd5eb3747cc2981ea4a0e4f8b6 WatchSource:0}: Error finding container 05a1839b624b0741419c0ca2ec69ac35af8228cd5eb3747cc2981ea4a0e4f8b6: Status 404 returned error can't find the container with id 05a1839b624b0741419c0ca2ec69ac35af8228cd5eb3747cc2981ea4a0e4f8b6 Sep 05 06:58:11.106855 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:11.106824 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55889757f5-pxx66" event={"ID":"0737305e-ae0a-4c9e-aea1-44062c2c2980","Type":"ContainerStarted","Data":"1ea757d31bc8840721d031308e3659652a64c7c410e177f50c18ce2ea54c5137"} Sep 05 06:58:11.107197 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:11.106885 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55889757f5-pxx66" event={"ID":"0737305e-ae0a-4c9e-aea1-44062c2c2980","Type":"ContainerStarted","Data":"05a1839b624b0741419c0ca2ec69ac35af8228cd5eb3747cc2981ea4a0e4f8b6"} Sep 05 06:58:11.107197 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:11.106911 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 06:58:11.122976 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:11.122922 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55889757f5-pxx66" podStartSLOduration=1.688587966 podStartE2EDuration="2.122902009s" totalImagesPullingTime="434.314043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:58:11.122092291 +0000 UTC m=+1987.805597201" watchObservedRunningTime="2026-09-05 06:58:11.122902009 +0000 UTC m=+1987.806406917" Sep 05 06:58:22.114173 ip-10-0-142-24 kubenswrapper[2583]: I0905 06:58:22.114141 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55889757f5-pxx66" Sep 05 07:00:04.005889 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:00:04.005852 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 07:00:04.016148 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:00:04.016119 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 07:00:04.020230 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:00:04.020210 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 07:00:04.027944 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:00:04.027926 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c8w2h_8118f2be-bfb1-46de-8a34-80bb58e1a5e7/ovn-acl-logging/0.log" Sep 05 07:02:28.498935 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.498853 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5699887589-p8tcl"] Sep 05 07:02:28.502598 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.502574 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.516068 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.515968 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5699887589-p8tcl"] Sep 05 07:02:28.618667 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.618635 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b901128-8c32-466e-ad18-5d399e2b66ac-openshift-service-ca\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.618667 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.618680 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b901128-8c32-466e-ad18-5d399e2b66ac-maas-api-tls\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.619060 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.618708 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg759\" (UniqueName: \"kubernetes.io/projected/7b901128-8c32-466e-ad18-5d399e2b66ac-kube-api-access-vg759\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.619060 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.618855 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b901128-8c32-466e-ad18-5d399e2b66ac-metrics-certs\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.720207 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.720169 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b901128-8c32-466e-ad18-5d399e2b66ac-openshift-service-ca\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.720386 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.720215 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b901128-8c32-466e-ad18-5d399e2b66ac-maas-api-tls\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.720456 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.720391 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg759\" (UniqueName: \"kubernetes.io/projected/7b901128-8c32-466e-ad18-5d399e2b66ac-kube-api-access-vg759\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.720549 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.720501 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b901128-8c32-466e-ad18-5d399e2b66ac-metrics-certs\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.720937 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.720918 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b901128-8c32-466e-ad18-5d399e2b66ac-openshift-service-ca\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.722847 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.722823 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b901128-8c32-466e-ad18-5d399e2b66ac-metrics-certs\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.722986 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.722952 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b901128-8c32-466e-ad18-5d399e2b66ac-maas-api-tls\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.727862 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.727841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg759\" (UniqueName: \"kubernetes.io/projected/7b901128-8c32-466e-ad18-5d399e2b66ac-kube-api-access-vg759\") pod \"maas-api-5699887589-p8tcl\" (UID: \"7b901128-8c32-466e-ad18-5d399e2b66ac\") " pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:28.817037 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:28.816936 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:29.101824 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:29.101680 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5699887589-p8tcl"] Sep 05 07:02:29.104623 ip-10-0-142-24 kubenswrapper[2583]: W0905 07:02:29.104584 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b901128_8c32_466e_ad18_5d399e2b66ac.slice/crio-5f5dbb381a2d294b4c00299e0f49d34e4bd0517662fb5d4f58c44f7fc71b9244 WatchSource:0}: Error finding container 5f5dbb381a2d294b4c00299e0f49d34e4bd0517662fb5d4f58c44f7fc71b9244: Status 404 returned error can't find the container with id 5f5dbb381a2d294b4c00299e0f49d34e4bd0517662fb5d4f58c44f7fc71b9244 Sep 05 07:02:29.578037 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:29.578010 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" event={"ID":"7b901128-8c32-466e-ad18-5d399e2b66ac","Type":"ContainerStarted","Data":"5f5dbb381a2d294b4c00299e0f49d34e4bd0517662fb5d4f58c44f7fc71b9244"} Sep 05 07:02:30.585606 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:30.585564 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" event={"ID":"7b901128-8c32-466e-ad18-5d399e2b66ac","Type":"ContainerStarted","Data":"8ecb8a8486e31007fe62456e15de4f8ec8cee21a78089fc8a26f11ea34087e87"} Sep 05 07:02:30.586072 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:30.585728 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:30.604163 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:30.604114 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" podStartSLOduration=2.15511554 podStartE2EDuration="2.604100007s" totalImagesPullingTime="448.984467ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:02:30.601222801 +0000 UTC m=+2247.284727711" watchObservedRunningTime="2026-09-05 07:02:30.604100007 +0000 UTC m=+2247.287604942" Sep 05 07:02:36.596911 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:36.596882 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5699887589-p8tcl" Sep 05 07:02:43.970893 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:43.970854 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz"] Sep 05 07:02:43.971276 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:43.971164 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" podUID="5ec03923-58c6-4931-8f31-e186f366f284" containerName="maas-api" containerID="cri-o://879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc" gracePeriod=30 Sep 05 07:02:45.234145 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.234115 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 07:02:45.269998 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.269913 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-maas-api-tls\") pod \"5ec03923-58c6-4931-8f31-e186f366f284\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " Sep 05 07:02:45.269998 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.269968 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggh2g\" (UniqueName: \"kubernetes.io/projected/5ec03923-58c6-4931-8f31-e186f366f284-kube-api-access-ggh2g\") pod \"5ec03923-58c6-4931-8f31-e186f366f284\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " Sep 05 07:02:45.270221 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.270037 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-metrics-certs\") pod \"5ec03923-58c6-4931-8f31-e186f366f284\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " Sep 05 07:02:45.270221 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.270110 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec03923-58c6-4931-8f31-e186f366f284-openshift-service-ca\") pod \"5ec03923-58c6-4931-8f31-e186f366f284\" (UID: \"5ec03923-58c6-4931-8f31-e186f366f284\") " Sep 05 07:02:45.270550 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.270517 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ec03923-58c6-4931-8f31-e186f366f284-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5ec03923-58c6-4931-8f31-e186f366f284" (UID: "5ec03923-58c6-4931-8f31-e186f366f284"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 05 07:02:45.272190 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.272157 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5ec03923-58c6-4931-8f31-e186f366f284" (UID: "5ec03923-58c6-4931-8f31-e186f366f284"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 07:02:45.272293 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.272198 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ec03923-58c6-4931-8f31-e186f366f284-kube-api-access-ggh2g" (OuterVolumeSpecName: "kube-api-access-ggh2g") pod "5ec03923-58c6-4931-8f31-e186f366f284" (UID: "5ec03923-58c6-4931-8f31-e186f366f284"). InnerVolumeSpecName "kube-api-access-ggh2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 05 07:02:45.272450 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.272417 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5ec03923-58c6-4931-8f31-e186f366f284" (UID: "5ec03923-58c6-4931-8f31-e186f366f284"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 05 07:02:45.371350 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.371316 2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ec03923-58c6-4931-8f31-e186f366f284-openshift-service-ca\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 07:02:45.371350 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.371344 2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-maas-api-tls\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 07:02:45.371350 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.371353 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggh2g\" (UniqueName: \"kubernetes.io/projected/5ec03923-58c6-4931-8f31-e186f366f284-kube-api-access-ggh2g\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 07:02:45.371350 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.371361 2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ec03923-58c6-4931-8f31-e186f366f284-metrics-certs\") on node \"ip-10-0-142-24.ec2.internal\" DevicePath \"\"" Sep 05 07:02:45.672294 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.672208 2583 generic.go:356] "Generic (PLEG): container finished" podID="5ec03923-58c6-4931-8f31-e186f366f284" containerID="879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc" exitCode=0 Sep 05 07:02:45.672294 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.672255 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" Sep 05 07:02:45.672294 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.672286 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" event={"ID":"5ec03923-58c6-4931-8f31-e186f366f284","Type":"ContainerDied","Data":"879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc"} Sep 05 07:02:45.672618 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.672361 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz" event={"ID":"5ec03923-58c6-4931-8f31-e186f366f284","Type":"ContainerDied","Data":"9da3f35b6c91797b3a224a05894e6f98356f2c7582f04df61a013180cd358f4d"} Sep 05 07:02:45.672618 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.672383 2583 scope.go:117] "RemoveContainer" containerID="879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc" Sep 05 07:02:45.682625 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.682609 2583 scope.go:117] "RemoveContainer" containerID="879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc" Sep 05 07:02:45.682903 ip-10-0-142-24 kubenswrapper[2583]: E0905 07:02:45.682886 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc\": container with ID starting with 879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc not found: ID does not exist" containerID="879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc" Sep 05 07:02:45.682951 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.682910 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc"} err="failed to get container status \"879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc\": rpc error: code = NotFound desc = could not find container \"879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc\": container with ID starting with 879c05d789f0dfdcce963cb8ab0c729fe5ae3d5d643807507659aaa969db88cc not found: ID does not exist" Sep 05 07:02:45.694178 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.694147 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz"] Sep 05 07:02:45.697872 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.697851 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-644985f7bf-hb5hz"] Sep 05 07:02:45.823989 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:02:45.823961 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ec03923-58c6-4931-8f31-e186f366f284" path="/var/lib/kubelet/pods/5ec03923-58c6-4931-8f31-e186f366f284/volumes" Sep 05 07:03:06.452632 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:06.452601 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-55889757f5-pxx66_0737305e-ae0a-4c9e-aea1-44062c2c2980/manager/0.log" Sep 05 07:03:06.696858 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:06.696811 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dl59m_b5058292-d2de-4800-90f9-f87a10f411f6/manager/0.log" Sep 05 07:03:07.088287 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:07.088259 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665f9d595b-9wgfd_10616a9b-2025-47d2-8f21-9334f0aefbdc/copy-manifests/0.log" Sep 05 07:03:07.094690 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:07.094667 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665f9d595b-9wgfd_10616a9b-2025-47d2-8f21-9334f0aefbdc/manager/0.log" Sep 05 07:03:07.464074 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:07.463996 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5699887589-p8tcl_7b901128-8c32-466e-ad18-5d399e2b66ac/maas-api/0.log" Sep 05 07:03:07.929923 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:07.929891 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-j4j25_e0ff1db7-eeb9-4315-a403-cf87b73f3b43/postgres/0.log" Sep 05 07:03:10.069080 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:10.069044 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g8pgk_370e8eb8-8963-4705-b5d6-62f22eb84326/discovery/0.log" Sep 05 07:03:11.155561 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:11.155529 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-tqvff_569eecc6-3781-4b66-b3ce-9b119ee1a87e/main/0.log" Sep 05 07:03:11.270094 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:11.270066 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mxfgg_3f0ca2da-42ed-4b6e-b663-9a14527a46a4/main/0.log" Sep 05 07:03:20.770467 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:20.770438 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vcd6d_93d036de-ea2d-406d-a61d-4f6c972acc71/global-pull-secret-syncer/0.log" Sep 05 07:03:20.849760 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:20.849731 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kvx7p_5d8ad0f2-e276-402b-8270-c6194aa5270f/konnectivity-agent/0.log" Sep 05 07:03:20.966063 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:20.966031 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-24.ec2.internal_36b5895e28f6f5810ea58bd85d6e692d/haproxy/0.log" Sep 05 07:03:27.014697 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.014666 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c71c9f8-6b23-4ac4-9d26-4195100e246d/alertmanager/0.log" Sep 05 07:03:27.034943 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.034920 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c71c9f8-6b23-4ac4-9d26-4195100e246d/config-reloader/0.log" Sep 05 07:03:27.057607 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.057585 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c71c9f8-6b23-4ac4-9d26-4195100e246d/kube-rbac-proxy-web/0.log" Sep 05 07:03:27.078460 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.078439 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c71c9f8-6b23-4ac4-9d26-4195100e246d/kube-rbac-proxy/0.log" Sep 05 07:03:27.098910 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.098887 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c71c9f8-6b23-4ac4-9d26-4195100e246d/kube-rbac-proxy-metric/0.log" Sep 05 07:03:27.118755 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.118733 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c71c9f8-6b23-4ac4-9d26-4195100e246d/prom-label-proxy/0.log" Sep 05 07:03:27.139386 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.139366 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6c71c9f8-6b23-4ac4-9d26-4195100e246d/init-config-reloader/0.log" Sep 05 07:03:27.179618 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.179595 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-6r7ml_2c15fd53-f580-45ab-aacf-5a21e7500817/cluster-monitoring-operator/0.log" Sep 05 07:03:27.204812 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.204789 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-r25d6_276312ef-5e4f-4a22-ae1c-600a2885e110/kube-state-metrics/0.log" Sep 05 07:03:27.220375 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.220357 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-r25d6_276312ef-5e4f-4a22-ae1c-600a2885e110/kube-rbac-proxy-main/0.log" Sep 05 07:03:27.240049 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.240026 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-r25d6_276312ef-5e4f-4a22-ae1c-600a2885e110/kube-rbac-proxy-self/0.log" Sep 05 07:03:27.396507 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.396395 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnd7c_0e3bf338-be1e-4459-ac3e-a372a1696832/node-exporter/0.log" Sep 05 07:03:27.415604 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.415572 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnd7c_0e3bf338-be1e-4459-ac3e-a372a1696832/kube-rbac-proxy/0.log" Sep 05 07:03:27.434201 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.434181 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnd7c_0e3bf338-be1e-4459-ac3e-a372a1696832/init-textfile/0.log" Sep 05 07:03:27.768646 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.768617 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-fwzxk_a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451/prometheus-operator/0.log" Sep 05 07:03:27.786538 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.786509 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-fwzxk_a3bb8608-d9aa-4bbc-8e42-8e22dd3e0451/kube-rbac-proxy/0.log" Sep 05 07:03:27.811875 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.811855 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-5cq9h_32e578ed-c724-41ac-9734-d82e91cfefb2/prometheus-operator-admission-webhook/0.log" Sep 05 07:03:27.841734 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.841713 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b47c7d785-t5d9g_47e4b646-b3ad-448c-b934-278d1b7b2d59/telemeter-client/0.log" Sep 05 07:03:27.868732 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.868711 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b47c7d785-t5d9g_47e4b646-b3ad-448c-b934-278d1b7b2d59/reload/0.log" Sep 05 07:03:27.889493 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.889457 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-b47c7d785-t5d9g_47e4b646-b3ad-448c-b934-278d1b7b2d59/kube-rbac-proxy/0.log" Sep 05 07:03:27.934347 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.934312 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/thanos-query/0.log" Sep 05 07:03:27.953076 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.953058 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/kube-rbac-proxy-web/0.log" Sep 05 07:03:27.972633 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.972602 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/kube-rbac-proxy/0.log" Sep 05 07:03:27.991378 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:27.991354 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/prom-label-proxy/0.log" Sep 05 07:03:28.010947 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:28.010928 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/kube-rbac-proxy-rules/0.log" Sep 05 07:03:28.031983 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:28.031927 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bff8865f5-6glhx_b8999492-9491-473c-a6c1-4e9ca9ee9efe/kube-rbac-proxy-metrics/0.log" Sep 05 07:03:29.728435 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:29.728399 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-fwvvg_d115bb76-6396-40b9-b87a-1e1ab8d31282/networking-console-plugin/0.log" Sep 05 07:03:30.363999 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.363953 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/2.log" Sep 05 07:03:30.368336 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.368317 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-8bx7z_a6b9ff6f-b0f0-43eb-8516-1939b0a95609/console-operator/3.log" Sep 05 07:03:30.911072 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.911037 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr"] Sep 05 07:03:30.911447 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.911429 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ec03923-58c6-4931-8f31-e186f366f284" containerName="maas-api" Sep 05 07:03:30.911447 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.911439 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ec03923-58c6-4931-8f31-e186f366f284" containerName="maas-api" Sep 05 07:03:30.911552 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.911538 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ec03923-58c6-4931-8f31-e186f366f284" containerName="maas-api" Sep 05 07:03:30.914876 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.914853 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:30.917735 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.917716 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q9qgh\"/\"default-dockercfg-ssk9n\"" Sep 05 07:03:30.917858 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.917717 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q9qgh\"/\"openshift-service-ca.crt\"" Sep 05 07:03:30.918390 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.918374 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q9qgh\"/\"kube-root-ca.crt\"" Sep 05 07:03:30.924127 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.924103 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr"] Sep 05 07:03:30.926820 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.926800 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-65c4f6986c-k7gfq_4f983edf-1d0b-493d-8368-422ac363e9ea/console/0.log" Sep 05 07:03:30.959346 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.959320 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-k9vlh_367e1810-ce3c-45ab-859c-80a508dab669/download-server/0.log" Sep 05 07:03:30.993212 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.993188 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-sys\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:30.993343 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.993231 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-proc\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:30.993343 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.993267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trzzv\" (UniqueName: \"kubernetes.io/projected/9ea33881-708b-41f0-89f7-72f779c3c58c-kube-api-access-trzzv\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:30.993343 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.993285 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-lib-modules\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:30.993343 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:30.993302 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-podres\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.093842 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.093809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-proc\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.093995 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.093851 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trzzv\" (UniqueName: \"kubernetes.io/projected/9ea33881-708b-41f0-89f7-72f779c3c58c-kube-api-access-trzzv\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.093995 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.093869 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-lib-modules\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.093995 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.093898 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-podres\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.093995 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.093932 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-proc\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.093995 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.093959 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-sys\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.094168 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.094001 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-lib-modules\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.094168 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.094018 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-podres\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.094168 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.094048 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ea33881-708b-41f0-89f7-72f779c3c58c-sys\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.101454 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.101422 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trzzv\" (UniqueName: \"kubernetes.io/projected/9ea33881-708b-41f0-89f7-72f779c3c58c-kube-api-access-trzzv\") pod \"perf-node-gather-daemonset-4s9tr\" (UID: \"9ea33881-708b-41f0-89f7-72f779c3c58c\") " pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.226225 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.226141 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.498079 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.498050 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr"] Sep 05 07:03:31.499643 ip-10-0-142-24 kubenswrapper[2583]: W0905 07:03:31.499616 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9ea33881_708b_41f0_89f7_72f779c3c58c.slice/crio-be3b21fb0a864faef91660750a5a3177ac276c74aece0fc9282d836c5b235362 WatchSource:0}: Error finding container be3b21fb0a864faef91660750a5a3177ac276c74aece0fc9282d836c5b235362: Status 404 returned error can't find the container with id be3b21fb0a864faef91660750a5a3177ac276c74aece0fc9282d836c5b235362 Sep 05 07:03:31.630987 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.630964 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-4nzw5_065e72da-226b-42f0-a35d-def84ac2743a/volume-data-source-validator/0.log" Sep 05 07:03:31.936185 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.936099 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" event={"ID":"9ea33881-708b-41f0-89f7-72f779c3c58c","Type":"ContainerStarted","Data":"f3a0e9650f0e526148097a6801e5ac3e3a34950a38b0f9a02893836244becd11"} Sep 05 07:03:31.936185 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.936150 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" event={"ID":"9ea33881-708b-41f0-89f7-72f779c3c58c","Type":"ContainerStarted","Data":"be3b21fb0a864faef91660750a5a3177ac276c74aece0fc9282d836c5b235362"} Sep 05 07:03:31.936612 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.936215 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:31.958106 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:31.958053 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" podStartSLOduration=1.9580385059999998 podStartE2EDuration="1.958038506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:03:31.956652656 +0000 UTC m=+2308.640157565" watchObservedRunningTime="2026-09-05 07:03:31.958038506 +0000 UTC m=+2308.641543412" Sep 05 07:03:32.844876 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:32.844847 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p8fq5_a3fff948-476a-4e52-824c-62d88e038d9d/dns/0.log" Sep 05 07:03:32.865995 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:32.865974 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p8fq5_a3fff948-476a-4e52-824c-62d88e038d9d/kube-rbac-proxy/0.log" Sep 05 07:03:32.983888 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:32.983862 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m42bc_302f2dc8-13ec-46b9-b694-f66048d4e2c3/dns-node-resolver/0.log" Sep 05 07:03:33.622970 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:33.622939 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4bf69_f64eb380-73df-46a4-9766-dd8d99bc4625/node-ca/0.log" Sep 05 07:03:34.837029 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:34.836994 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g8pgk_370e8eb8-8963-4705-b5d6-62f22eb84326/discovery/0.log" Sep 05 07:03:35.892459 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:35.892426 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hwz4f_2582be7e-4f6c-4fdd-b852-2bed0a6d8cf4/serve-healthcheck-canary/0.log" Sep 05 07:03:36.454201 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:36.454172 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b29nx_49f311ef-2d4d-4cf1-93e4-4138b14525a5/kube-rbac-proxy/0.log" Sep 05 07:03:36.472837 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:36.472809 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b29nx_49f311ef-2d4d-4cf1-93e4-4138b14525a5/exporter/0.log" Sep 05 07:03:36.492770 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:36.492744 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b29nx_49f311ef-2d4d-4cf1-93e4-4138b14525a5/extractor/0.log" Sep 05 07:03:37.951304 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:37.951276 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q9qgh/perf-node-gather-daemonset-4s9tr" Sep 05 07:03:38.891896 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:38.891863 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-55889757f5-pxx66_0737305e-ae0a-4c9e-aea1-44062c2c2980/manager/0.log" Sep 05 07:03:38.991347 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:38.991318 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dl59m_b5058292-d2de-4800-90f9-f87a10f411f6/manager/0.log" Sep 05 07:03:39.178410 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:39.178335 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665f9d595b-9wgfd_10616a9b-2025-47d2-8f21-9334f0aefbdc/manager/0.log" Sep 05 07:03:39.197395 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:39.197369 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665f9d595b-9wgfd_10616a9b-2025-47d2-8f21-9334f0aefbdc/copy-manifests/0.log" Sep 05 07:03:40.601175 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:40.601106 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-688d76797b-7m4d4_56b0a15b-232a-4e7f-8c4f-ac45619c11e0/manager/0.log" Sep 05 07:03:41.575831 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:41.575799 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g8pgk_370e8eb8-8963-4705-b5d6-62f22eb84326/discovery/0.log" Sep 05 07:03:43.480339 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:43.480302 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g8pgk_370e8eb8-8963-4705-b5d6-62f22eb84326/discovery/0.log" Sep 05 07:03:45.788144 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:45.788110 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-4f2jb_5710c05e-19b5-486d-8697-1cde26e40a86/migrator/0.log" Sep 05 07:03:45.812847 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:45.812818 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-4f2jb_5710c05e-19b5-486d-8697-1cde26e40a86/graceful-termination/0.log" Sep 05 07:03:47.095485 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:47.095458 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7tt2w_277802fb-ac81-406a-83aa-2824daafcb00/kube-multus/0.log" Sep 05 07:03:47.260366 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:47.260339 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kglkm_ba4875c4-1642-481c-88b7-f59c243040e4/kube-multus-additional-cni-plugins/0.log" Sep 05 07:03:47.280655 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:47.280630 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kglkm_ba4875c4-1642-481c-88b7-f59c243040e4/egress-router-binary-copy/0.log" Sep 05 07:03:47.299468 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:47.299448 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kglkm_ba4875c4-1642-481c-88b7-f59c243040e4/cni-plugins/0.log" Sep 05 07:03:47.318535 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:47.318516 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kglkm_ba4875c4-1642-481c-88b7-f59c243040e4/bond-cni-plugin/0.log" Sep 05 07:03:47.338176 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:47.338154 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kglkm_ba4875c4-1642-481c-88b7-f59c243040e4/routeoverride-cni/0.log" Sep 05 07:03:47.356801 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:47.356744 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kglkm_ba4875c4-1642-481c-88b7-f59c243040e4/whereabouts-cni-bincopy/0.log" Sep 05 07:03:47.382632 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:47.382610 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kglkm_ba4875c4-1642-481c-88b7-f59c243040e4/whereabouts-cni/0.log" Sep 05 07:03:47.703165 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:47.703080 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-m8fn4_68080680-0551-487c-80ed-b93acb2f68f8/network-metrics-daemon/0.log" Sep 05 07:03:47.720629 ip-10-0-142-24 kubenswrapper[2583]: I0905 07:03:47.720603 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-m8fn4_68080680-0551-487c-80ed-b93acb2f68f8/kube-rbac-proxy/0.log"