Sep 11 00:55:58.580177 ip-10-0-138-14 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 00:55:58.580187 ip-10-0-138-14 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 00:55:58.580197 ip-10-0-138-14 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 00:55:58.580502 ip-10-0-138-14 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 00:56:08.596984 ip-10-0-138-14 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 00:56:08.597002 ip-10-0-138-14 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d1aea9d08a4c43ac87264908c36d1fe0 --
Sep 11 00:58:00.366960 ip-10-0-138-14 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 00:58:00.787269 ip-10-0-138-14 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 00:58:00.787269 ip-10-0-138-14 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 00:58:00.787269 ip-10-0-138-14 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 00:58:00.787269 ip-10-0-138-14 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 00:58:00.787269 ip-10-0-138-14 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 00:58:00.788261 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.788172    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 00:58:00.791954 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791937    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:00.791954 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791954    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791958    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791962    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791965    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791968    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791971    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791974    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791977    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791979    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791982    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791985    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791988    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791990    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791993    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791996    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.791998    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792003    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792006    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792010    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:00.792023 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792012    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792015    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792018    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792020    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792023    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792025    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792028    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792031    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792037    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792040    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792042    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792045    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792048    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792051    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792053    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792056    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792059    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792062    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792065    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792067    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:00.792479 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792070    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792072    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792075    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792077    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792080    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792082    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792085    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792088    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792091    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792093    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792096    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792098    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792101    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792104    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792107    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792110    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792112    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792115    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792117    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792120    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:00.792987 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792123    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792125    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792128    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792130    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792133    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792135    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792138    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792140    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792143    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792146    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792148    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792151    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792153    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792156    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792158    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792161    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792163    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792165    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792169    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792172    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:00.793486 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792175    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792177    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792180    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792182    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792185    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792188    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792192    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792194    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792197    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792200    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792202    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792205    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792207    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792210    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792212    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792216    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792220    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792223    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792225    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:00.793980 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792988    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792996    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.792999    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793002    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793006    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793010    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793013    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793016    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793019    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793023    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793026    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793028    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793031    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793034    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793044    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793047    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793049    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793053    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793058    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:00.794438 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793061    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793064    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793067    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793070    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793072    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793075    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793077    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793080    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793083    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793085    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793088    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793090    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793093    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793096    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793098    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793100    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793103    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793105    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793109    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:00.794960 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793113    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793116    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793119    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793122    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793125    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793127    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793130    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793133    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793135    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793138    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793141    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793143    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793146    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793149    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793152    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793154    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793157    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793159    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793162    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793165    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793167    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:00.795440 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793169    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793172    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793174    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793177    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793179    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793182    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793185    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793187    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793189    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793192    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793195    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793198    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793200    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793203    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793206    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793208    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793211    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793213    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793218    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793220    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:00.796060 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793224    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793227    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793230    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793233    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793236    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793239    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793241    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793244    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793246    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793249    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793252    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793254    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793257    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793259    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793262    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793264    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793267    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793269    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793272    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:00.796547 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793274    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793712    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793719    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793723    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793726    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793729    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793732    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793734    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793737    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793740    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793742    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793745    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793747    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793751    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793756    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793759    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793762    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793765    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793768    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:00.797034 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793770    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793773    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793775    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793778    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793780    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793783    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793785    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793788    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793790    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793793    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793795    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793798    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793800    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793803    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793805    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793810    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793813    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793816    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793819    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793822    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:00.797499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793825    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793828    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793831    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793833    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793836    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793838    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793841    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793843    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793854    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793857    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793859    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793862    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793864    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793867    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793869    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793872    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793874    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793877    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793879    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793882    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:00.798010 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793885    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793887    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793889    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793892    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793895    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793897    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793900    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793902    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793906    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793908    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793911    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793913    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793916    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793919    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793921    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793924    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793926    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793930    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793933    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793935    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:00.798499 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793939    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793945    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793948    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793950    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793953    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793956    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793958    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793961    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793963    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793966    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793968    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793970    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793973    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793975    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793978    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793980    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793983    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793985    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793988    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793992    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:00.799005 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.793996    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794068    2559 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794102    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794114    2559 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794118    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794123    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794126    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794131    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794135    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794138    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794141    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794145    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794150    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794153    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794157    2559 flags.go:64] FLAG: --cgroup-root=""
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794160    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794163    2559 flags.go:64] FLAG: --client-ca-file=""
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794166    2559 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794170    2559 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794174    2559 flags.go:64] FLAG: --cluster-domain=""
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794177    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794180    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794183    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794187    2559 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 00:58:00.799534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794190    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794193    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794196    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794199    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794202    2559 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794205    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794208    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794211    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794216    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794220    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794223    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794226    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794229    2559 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794232    2559 flags.go:64] FLAG: --enable-server="true"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794235    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794240    2559 flags.go:64] FLAG: --event-burst="100"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794243    2559 flags.go:64] FLAG: --event-qps="50"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794246    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794249    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794252    2559 flags.go:64] FLAG: --eviction-hard=""
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794257    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794260    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794263    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794267    2559 flags.go:64] FLAG: --eviction-soft=""
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794270    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 00:58:00.800149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794273    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794276    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794279    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794282    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794286    2559 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794289    2559 flags.go:64] FLAG: --feature-gates=""
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794293    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794296    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794299    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794302    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794305    2559 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794308    2559 flags.go:64] FLAG: --help="false"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794311    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-138-14.ec2.internal"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794314    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794317    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794321    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794324    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794328    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794331    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794333    2559 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794336    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794339    2559 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794342    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794346    2559 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 00:58:00.800772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794348    2559 flags.go:64] FLAG: --kube-reserved=""
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794352    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794354    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794358    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794361    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794364    2559 flags.go:64] FLAG: --lock-file=""
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794367    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794376    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794379    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794384    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794387    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794390    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794393    2559 flags.go:64] FLAG: --logging-format="text"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794396    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794399    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794402    2559 flags.go:64] FLAG: --manifest-url=""
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794405    2559 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794410    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794413    2559 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794417    2559 flags.go:64] FLAG: --max-pods="110"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794420    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794422    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794425    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794430    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794433    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 00:58:00.801364 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794435    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794438    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794446    2559 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794450    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794453    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794456    2559 flags.go:64] FLAG: --pod-cidr=""
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794459    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794465    2559 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794468    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794471    2559 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794474    2559 flags.go:64] FLAG: --port="10250"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794478    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794481    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a357a1369761a8d2"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794484    2559 flags.go:64] FLAG: --qos-reserved=""
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794487    2559 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794497    2559 flags.go:64] FLAG: --register-node="true"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794500    2559 flags.go:64] FLAG: --register-with-taints=""
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794504    2559 flags.go:64] FLAG: --registry-burst="10"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794507    2559 flags.go:64] FLAG: --registry-qps="5"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794510    2559 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794513    2559 flags.go:64] FLAG: --reserved-memory=""
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794516    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794519    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794522    2559 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794525    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 00:58:00.801996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794528    2559 flags.go:64] FLAG: --runonce="false"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794531    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794534    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794537    2559 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794540    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794543    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794547    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794550    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794553    2559 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794556    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794564    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794567    2559 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794570    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794573    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794576    2559 flags.go:64] FLAG: --system-cgroups=""
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794579    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794584    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794587    2559 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794591    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794595    2559 flags.go:64] FLAG: --tls-min-version=""
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794598    2559 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794613    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794616    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794620    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794624    2559 flags.go:64] FLAG: --v="2"
Sep 11 00:58:00.802612 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794628    2559 flags.go:64] FLAG: --version="false"
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794633    2559 flags.go:64] FLAG: --vmodule=""
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794636    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.794640    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794759    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794763    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794766    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794769    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794772    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794775    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794778    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794781    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794783    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794787    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794790    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794793    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794796    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794798    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794801    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794804    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794806    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:00.803276 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794809    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794811    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794814    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794817    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794820    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794822    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794825    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794828    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794830    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794833    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794836    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794839    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794843    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794846    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794849    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794852    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794855    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794858    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794860    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794863    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:00.803819 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794866    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794868    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794871    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794873    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794878    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794881    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794885    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794888    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794890    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794893    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794896    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794898    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794901    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794903    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794906    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794909    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794911    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794914    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794917    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:00.804308 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794919    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794922    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794925    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794927    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794933    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794936    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794938    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794941    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794943    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794946    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794948    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794951    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794953    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794956    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794958    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794961    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794963    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794968    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794970    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794973    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:00.804809 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794976    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794978    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794983    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794986    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794989    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794992    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794995    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.794999    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795002    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795004    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795007    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795009    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795012    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795014    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795017    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795019    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795022    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795025    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795027    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:00.805293 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795030    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795032    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795035    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.795038    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.795916    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.803297    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.803317    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.803338    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.803347    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803409    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803415    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803418    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803423    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803427    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803431    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:00.805770 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803434    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803437    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803440    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803443    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803445    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803448    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803451    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803453    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803456    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803458    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803461    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803463    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803466    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803469    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803471    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803474    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803476    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803479    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803481    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:00.806149 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803484    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803487    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803489    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803492    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803502    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803505    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803508    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803510    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803513    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803515    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803518    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803521    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803524    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803526    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803529    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803531    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803534    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803537    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803539    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803542    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:00.806652 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803544    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803547    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803549    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803552    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803555    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803557    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803560    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803562    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803565    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803567    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803570    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803572    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803575    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803578    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803580    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803583    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803588    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803597    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803614    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803619    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:00.807144 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803622    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803625    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803628    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803630    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803633    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803636    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803638    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803641    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803644    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803646    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803649    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803651    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803654    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803658    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803661    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803665    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803667    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803670    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803672    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803675    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:00.807654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803677    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803680    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803683    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803685    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803688    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803690    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803693    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803696    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803699    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803702    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803710    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803713    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803716    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803719    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.803723    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 00:58:00.808153 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803879    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803885    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803888    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803891    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803894    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803897    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803899    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803902    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803906    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803910    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803913    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803916    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803919    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803921    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803941    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803945    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803948    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803950    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803953    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803956    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 00:58:00.808521 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803959    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803961    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803964    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803966    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803969    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803972    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803975    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803978    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803986    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803989    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803992    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803994    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803997    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.803999    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804002    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804004    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804007    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804009    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804012    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804015    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 00:58:00.809118 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804017    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804020    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804022    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804024    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804027    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804029    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804032    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804034    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804037    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804040    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804042    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804045    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804047    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804050    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804053    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804055    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804058    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804060    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804063    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804065    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 00:58:00.809624 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804069    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804078    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804081    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804084    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804087    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804090    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804093    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804095    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804098    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804100    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804103    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804105    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804108    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804110    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804113    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804115    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804118    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804120    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804123    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 00:58:00.810110 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804126    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804128    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804131    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804133    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804136    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804138    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804141    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804143    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804146    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804149    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804152    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804156    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804158    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804161    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804164    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804173    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804175    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804178    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804180    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 00:58:00.810575 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:00.804183    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 00:58:00.811039 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.804187    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 00:58:00.811039 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.804331    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 00:58:00.811039 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.807319    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 00:58:00.811039 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.808400    2559 server.go:1019] "Starting client certificate rotation"
Sep 11 00:58:00.811039 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.808494    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 00:58:00.811039 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.808535    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 00:58:00.811792 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.811778    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 00:58:00.815889 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.814428    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 00:58:00.824875 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.824855    2559 log.go:25] "Validated CRI v1 runtime API"
Sep 11 00:58:00.829010 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.828992    2559 log.go:25] "Validated CRI v1 image API"
Sep 11 00:58:00.829904 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.829891    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 00:58:00.834928 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.834906    2559 fs.go:135] Filesystem UUIDs: map[4b069d8d-33fa-4410-9b23-765f71c5100f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8ff368d6-517e-48b6-8abc-f4c9ffc12609:/dev/nvme0n1p4]
Sep 11 00:58:00.835012 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.834925    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 00:58:00.839992 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.839975    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 00:58:00.841260 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.841141    2559 manager.go:216] Machine: {Timestamp:2026-09-11 00:58:00.839701654 +0000 UTC m=+0.354301826 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099958 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2088b76637d46b059d30582b6a840c SystemUUID:ec2088b7-6637-d46b-059d-30582b6a840c BootID:d1aea9d0-8a4c-43ac-8726-4908c36d1fe0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:04:ad:2b:d2:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:04:ad:2b:d2:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:37:0a:9a:08:0e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 00:58:00.841260 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.841260    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 00:58:00.841360 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.841348    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 00:58:00.845126 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.845098    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 00:58:00.845269 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.845128    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-14.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 00:58:00.846812 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.846801    2559 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 00:58:00.846848 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.846815    2559 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 00:58:00.846848 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.846823    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 00:58:00.847428 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.847418    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 00:58:00.848302 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.848290    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 00:58:00.851446 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.851433    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 00:58:00.851565 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.851555    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 00:58:00.851594 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.851585    2559 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 00:58:00.851646 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.851594    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 00:58:00.851646 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.851620    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 00:58:00.851646 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.851631    2559 kubelet.go:398] "Adding apiserver pod source"
Sep 11 00:58:00.851646 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.851640    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 00:58:00.852083 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.852070    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 00:58:00.852151 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.852090    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 00:58:00.856174 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.856152    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 00:58:00.858003 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.857989    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 00:58:00.858092 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.858020    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 00:58:00.859765 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859754    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 00:58:00.859808 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859769    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 00:58:00.859808 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859775    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 00:58:00.859808 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859780    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 00:58:00.859808 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859789    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 00:58:00.859808 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859807    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 00:58:00.859941 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859822    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 00:58:00.859941 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859827    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 00:58:00.859941 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859837    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 00:58:00.859941 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859842    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 00:58:00.859941 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859851    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 00:58:00.859941 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.859856    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 00:58:00.860635 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.860622    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 00:58:00.860674 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.860637    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 00:58:00.864168 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.864143    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8w5tl"
Sep 11 00:58:00.864537 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.864523    2559 server.go:1268] "Started kubelet"
Sep 11 00:58:00.865489 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.865446    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 00:58:00.865577 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.865530    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 00:58:00.866232 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.866206    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-14.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 00:58:00.866330 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:00.866212    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 00:58:00.866330 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.865448    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 00:58:00.866330 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:00.866273    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 00:58:00.866497 ip-10-0-138-14 systemd[1]: Started Kubernetes Kubelet.
Sep 11 00:58:00.868190 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.868167    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 00:58:00.870180 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.870161    2559 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 00:58:00.871283 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.871261    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8w5tl"
Sep 11 00:58:00.872313 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:00.872293    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 00:58:00.875126 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.875106    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 00:58:00.875657 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.875640    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 00:58:00.875755 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:00.875737    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:00.876134 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:00.874437    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-14.ec2.internal.18d41e4ffcbe0497  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-14.ec2.internal,UID:ip-10-0-138-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-14.ec2.internal,},FirstTimestamp:2026-09-11 00:58:00.864490647 +0000 UTC m=+0.379090859,LastTimestamp:2026-09-11 00:58:00.864490647 +0000 UTC m=+0.379090859,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-14.ec2.internal,}"
Sep 11 00:58:00.876233 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876164    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 00:58:00.876277 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876265    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 00:58:00.876322 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876281    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 00:58:00.876423 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876407    2559 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 00:58:00.876423 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876420    2559 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 00:58:00.876554 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876525    2559 factory.go:55] Registering systemd factory
Sep 11 00:58:00.876554 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876541    2559 factory.go:223] Registration of the systemd container factory successfully
Sep 11 00:58:00.876755 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876741    2559 factory.go:153] Registering CRI-O factory
Sep 11 00:58:00.876816 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876759    2559 factory.go:223] Registration of the crio container factory successfully
Sep 11 00:58:00.876816 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876812    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 00:58:00.876902 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876831    2559 factory.go:103] Registering Raw factory
Sep 11 00:58:00.876902 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.876845    2559 manager.go:1196] Started watching for new ooms in manager
Sep 11 00:58:00.877185 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.877172    2559 manager.go:349] Starting recovery of all containers
Sep 11 00:58:00.878210 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.878181    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:00.881151 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:00.881127    2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-14.ec2.internal\" not found" node="ip-10-0-138-14.ec2.internal"
Sep 11 00:58:00.887238 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.886967    2559 manager.go:354] Recovery completed
Sep 11 00:58:00.892004 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.891986    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:00.895827 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.895807    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:00.895924 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.895840    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:00.895924 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.895868    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:00.896531 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.896512    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 00:58:00.896531 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.896523    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:00.896531 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.896526    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 00:58:00.896688 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.896545    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 00:58:00.897277 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.897259    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:00.897356 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.897286    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:00.897356 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.897300    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:00.897836 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.897821    2559 policy_none.go:49] "None policy: Start"
Sep 11 00:58:00.897899 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.897840    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 00:58:00.897899 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:00.897854    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 00:58:00.976091 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:00.976043    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.047150 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.047050    2559 policy_none.go:47] "Start"
Sep 11 00:58:01.076926 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.076904    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.080175 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.080159    2559 manager.go:337] "Starting Device Plugin manager"
Sep 11 00:58:01.080267 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.080201    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 00:58:01.080267 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.080216    2559 server.go:87] "Starting device plugin registration server"
Sep 11 00:58:01.080591 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.080577    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 00:58:01.080677 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.080592    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 00:58:01.080725 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.080695    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 00:58:01.080791 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.080779    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 00:58:01.080791 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.080790    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 00:58:01.081400 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.081380    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 00:58:01.081460 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.081427    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.102034 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.102001    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 00:58:01.103145 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.103123    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 00:58:01.103224 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.103184    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 00:58:01.103224 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.103211    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 00:58:01.103288 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.103256    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 00:58:01.105496 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.105478    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:01.180746 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.180707    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:01.182351 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.182329    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:01.182444 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.182370    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:01.182444 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.182385    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:01.182444 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.182409    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.190689 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.190672    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.190736 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.190695    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-14.ec2.internal\": node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.203339 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.203309    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal"]
Sep 11 00:58:01.203477 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.203394    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:01.205351 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.205336    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:01.205427 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.205366    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:01.205427 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.205380    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:01.206655 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.206643    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:01.206771 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.206756    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.206809 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.206791    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:01.207331 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.207313    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.207453 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.207441    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:01.207518 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.207464    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:01.207518 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.207478    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:01.207631 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.207521    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:01.207631 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.207542    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:01.207631 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.207551    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:01.208742 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.208726    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.208823 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.208757    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 00:58:01.209402 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.209387    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 00:58:01.209483 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.209417    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 00:58:01.209483 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.209431    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeHasSufficientPID"
Sep 11 00:58:01.241392 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.241367    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-14.ec2.internal\" not found" node="ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.245652 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.245634    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-14.ec2.internal\" not found" node="ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.279432 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.279405    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/631b5c907d6ae04b2ef2175a9c73889c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal\" (UID: \"631b5c907d6ae04b2ef2175a9c73889c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.279432 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.279432    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/631b5c907d6ae04b2ef2175a9c73889c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal\" (UID: \"631b5c907d6ae04b2ef2175a9c73889c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.279635 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.279451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e42a195d24144df11416cc5ea52d2a36-config\") pod \"kube-apiserver-proxy-ip-10-0-138-14.ec2.internal\" (UID: \"e42a195d24144df11416cc5ea52d2a36\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.308280 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.308258    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.380129 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.380096    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/631b5c907d6ae04b2ef2175a9c73889c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal\" (UID: \"631b5c907d6ae04b2ef2175a9c73889c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.380129 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.380134    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e42a195d24144df11416cc5ea52d2a36-config\") pod \"kube-apiserver-proxy-ip-10-0-138-14.ec2.internal\" (UID: \"e42a195d24144df11416cc5ea52d2a36\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.380295 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.380150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/631b5c907d6ae04b2ef2175a9c73889c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal\" (UID: \"631b5c907d6ae04b2ef2175a9c73889c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.380295 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.380199    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e42a195d24144df11416cc5ea52d2a36-config\") pod \"kube-apiserver-proxy-ip-10-0-138-14.ec2.internal\" (UID: \"e42a195d24144df11416cc5ea52d2a36\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.380295 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.380206    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/631b5c907d6ae04b2ef2175a9c73889c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal\" (UID: \"631b5c907d6ae04b2ef2175a9c73889c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.380295 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.380246    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/631b5c907d6ae04b2ef2175a9c73889c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal\" (UID: \"631b5c907d6ae04b2ef2175a9c73889c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.410331 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.410295    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.512377 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.512331    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.544971 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.544939    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.548230 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.548210    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:01.613217 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.613120    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.714175 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.714131    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.809505 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.809469    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 00:58:01.810005 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.809665    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 00:58:01.810005 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.809716    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 00:58:01.815127 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.815106    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.874871 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.874788    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 00:53:01 +0000 UTC" deadline="2028-04-11 15:23:35.401843897 +0000 UTC"
Sep 11 00:58:01.874871 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.874820    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13886h25m33.5270279s"
Sep 11 00:58:01.916587 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:01.916543    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:01.930371 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:01.930345    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:02.017522 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:02.017485    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:02.118507 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:02.118467    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:02.144654 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:02.144615    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42a195d24144df11416cc5ea52d2a36.slice/crio-9d9afb91839d458ecb0ae0cf09a0e530ee40566c9941217c3a4e20adf2b7cc9d WatchSource:0}: Error finding container 9d9afb91839d458ecb0ae0cf09a0e530ee40566c9941217c3a4e20adf2b7cc9d: Status 404 returned error can't find the container with id 9d9afb91839d458ecb0ae0cf09a0e530ee40566c9941217c3a4e20adf2b7cc9d
Sep 11 00:58:02.144908 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:02.144887    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631b5c907d6ae04b2ef2175a9c73889c.slice/crio-44d4bbc6078947e219dd10ee41c14477708cd85b718cae5c599a21d078f93e91 WatchSource:0}: Error finding container 44d4bbc6078947e219dd10ee41c14477708cd85b718cae5c599a21d078f93e91: Status 404 returned error can't find the container with id 44d4bbc6078947e219dd10ee41c14477708cd85b718cae5c599a21d078f93e91
Sep 11 00:58:02.149286 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.149268    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 00:58:02.219940 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:02.219894    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-14.ec2.internal\" not found"
Sep 11 00:58:02.248993 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.248962    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:02.277360 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.277322    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:02.288038 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.288014    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 00:58:02.289214 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.289200    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal"
Sep 11 00:58:02.300406 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.300384    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 00:58:02.627698 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.627605    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:02.854037 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.853978    2559 apiserver.go:52] "Watching apiserver"
Sep 11 00:58:02.862708 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.862312    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 00:58:02.863315 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.862895    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-zndzf","kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal","openshift-image-registry/node-ca-hrx2c","openshift-multus/multus-52jsv","openshift-multus/multus-additional-cni-plugins-tgf62","openshift-multus/network-metrics-daemon-7qkdk","openshift-network-operator/iptables-alerter-nfqw9","openshift-ovn-kubernetes/ovnkube-node-lr2p4","kube-system/global-pull-secret-syncer-gg5rc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8","openshift-cluster-node-tuning-operator/tuned-rf5j7","openshift-dns/node-resolver-twtj4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal","openshift-network-diagnostics/network-check-target-v55tj"]
Sep 11 00:58:02.865562 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.865008    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:02.869419 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.867585    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.869419 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.868286    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 00:58:02.869419 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.868502    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 00:58:02.869419 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.868811    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 00:58:02.869419 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.868995    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qjflx\""
Sep 11 00:58:02.872794 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.870131    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 00:58:02.872794 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.870503    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 00:58:02.872794 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.870706    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ch2rg\""
Sep 11 00:58:02.872794 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.870905    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:02.872794 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.871547    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.872794 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.872386    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.874081 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.873345    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 00:58:02.874081 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.873350    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:02.874081 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.873522    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wwmd8\""
Sep 11 00:58:02.874081 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:02.873523    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:02.874081 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.873745    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 00:58:02.874081 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.873763    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 00:58:02.874081 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.873945    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 00:58:02.874571 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.874395    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 00:58:02.874697 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.874612    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-65s48\""
Sep 11 00:58:02.875472 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.874832    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 00:58:02.875472 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.874929    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 00:58:02.875472 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.875089    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 00:58:02.875472 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.875123    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l2lq6\""
Sep 11 00:58:02.875472 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.875305    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 00:58:02.875472 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.875429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:02.875865 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:02.875506    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:02.875865 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.875680    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:02.875865 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:02.875733    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:02.876095 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.876065    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 00:58:02.877426 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.877406    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.880733 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.880464    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:02.880733 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.880631    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 00:58:02.881817 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.881260    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 00:58:02.881817 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.881533    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 00:58:02.881817 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.881532    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 00:58:02.881817 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.881611    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 00:58:02.882481 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.881922    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 00:58:02.882481 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.882056    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8fv6z\""
Sep 11 00:58:02.884839 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.883078    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 00:58:02.884839 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.883293    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 00:58:02.884839 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.883592    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:02.884839 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.884219    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 00:58:02.885735 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.885664    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vgk7n\""
Sep 11 00:58:02.886344 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.886322    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:02.886742 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.886716    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-system-cni-dir\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.886925 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.886758    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-socket-dir-parent\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.886993 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.886944    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-conf-dir\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.886993 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.886974    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlw2b\" (UniqueName: \"kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b\") pod \"network-check-target-v55tj\" (UID: \"39e30de3-b555-45b2-84b5-0c93e875bddb\") " pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:02.887092 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887001    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94021732-9afe-4fc7-aad6-b71ebbed7af9-ovnkube-config\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.887092 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887026    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvpv9\" (UniqueName: \"kubernetes.io/projected/f3223957-508e-40d6-a6aa-6073e8438ca5-kube-api-access-lvpv9\") pod \"iptables-alerter-nfqw9\" (UID: \"f3223957-508e-40d6-a6aa-6073e8438ca5\") " pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:02.887092 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887050    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/820c3bfc-578f-41c3-97c1-5a5a356345a8-cni-binary-copy\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.887092 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887075    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd3fdc46-edd4-4833-baca-c730c52fed08-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.887256 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887103    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxcp4\" (UniqueName: \"kubernetes.io/projected/4d739055-efd4-41fe-9725-197913a5257c-kube-api-access-bxcp4\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:02.887256 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94021732-9afe-4fc7-aad6-b71ebbed7af9-env-overrides\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.887256 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887165    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-var-lib-cni-multus\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.887256 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887190    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd3fdc46-edd4-4833-baca-c730c52fed08-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.887256 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887222    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2484980c-48af-426f-b3c2-95afc18ed2a2-kubelet-config\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:02.887456 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887255    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2484980c-48af-426f-b3c2-95afc18ed2a2-dbus\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:02.887456 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887285    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-kubelet\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.887456 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887309    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-slash\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.887456 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887334    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-run-systemd\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.887456 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887363    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-run\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.887456 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887387    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-cni-netd\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.887456 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887417    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3223957-508e-40d6-a6aa-6073e8438ca5-host-slash\") pod \"iptables-alerter-nfqw9\" (UID: \"f3223957-508e-40d6-a6aa-6073e8438ca5\") " pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:02.887456 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvqdn\" (UniqueName: \"kubernetes.io/projected/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-kube-api-access-dvqdn\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887466    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-cni-dir\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887490    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887516    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3223957-508e-40d6-a6aa-6073e8438ca5-iptables-alerter-script\") pod \"iptables-alerter-nfqw9\" (UID: \"f3223957-508e-40d6-a6aa-6073e8438ca5\") " pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-host\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887598    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-hostroot\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887635    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-var-lib-openvswitch\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887660    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-run-openvswitch\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887704    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-run-ovn\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cn9g\" (UniqueName: \"kubernetes.io/projected/94021732-9afe-4fc7-aad6-b71ebbed7af9-kube-api-access-2cn9g\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887821    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-tmp\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887849    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-etc-kubernetes\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.887874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd3fdc46-edd4-4833-baca-c730c52fed08-cnibin\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887901    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-run-ovn-kubernetes\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-lib-modules\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887970    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-run-multus-certs\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.887995    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-systemd-units\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888020    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94021732-9afe-4fc7-aad6-b71ebbed7af9-ovnkube-script-lib\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888036    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888045    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-sysconfig\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888076    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-var-lib-kubelet\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888102    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd3fdc46-edd4-4833-baca-c730c52fed08-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888129    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-run-netns\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-node-log\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888179    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-modprobe-d\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888217    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-sys\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888242    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-systemd\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888264    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-tuned\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888274    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888288    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-cnibin\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.888396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888319    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-cni-bin\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-sysctl-conf\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888373    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-var-lib-cni-bin\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888404    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl2tc\" (UniqueName: \"kubernetes.io/projected/dd3fdc46-edd4-4833-baca-c730c52fed08-kube-api-access-kl2tc\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888430    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-etc-openvswitch\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888456    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-kubernetes\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888481    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a670085-8845-40b7-ab78-2d028806840c-host\") pod \"node-ca-hrx2c\" (UID: \"0a670085-8845-40b7-ab78-2d028806840c\") " pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888484    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ztsqp\""
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888697    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888719    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcfnm\" (UniqueName: \"kubernetes.io/projected/820c3bfc-578f-41c3-97c1-5a5a356345a8-kube-api-access-rcfnm\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a670085-8845-40b7-ab78-2d028806840c-serviceca\") pod \"node-ca-hrx2c\" (UID: \"0a670085-8845-40b7-ab78-2d028806840c\") " pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888778    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-var-lib-kubelet\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888802    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-daemon-config\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888829    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd3fdc46-edd4-4833-baca-c730c52fed08-system-cni-dir\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrcwx\" (UniqueName: \"kubernetes.io/projected/0a670085-8845-40b7-ab78-2d028806840c-kube-api-access-nrcwx\") pod \"node-ca-hrx2c\" (UID: \"0a670085-8845-40b7-ab78-2d028806840c\") " pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888883    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-run-k8s-cni-cncf-io\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-run-netns\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888938    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd3fdc46-edd4-4833-baca-c730c52fed08-cni-binary-copy\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.889268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.888977    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:02.890290 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.889004    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-log-socket\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.890290 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.889031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.890290 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.889065    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94021732-9afe-4fc7-aad6-b71ebbed7af9-ovn-node-metrics-cert\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.890290 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.889083    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-td8k7\""
Sep 11 00:58:02.890290 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.889089    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-sysctl-d\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.890290 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.889278    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 00:58:02.892741 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.892603    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 00:58:02.912682 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.912657    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fn8h8"
Sep 11 00:58:02.919684 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.919530    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fn8h8"
Sep 11 00:58:02.977400 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.977369    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 00:58:02.989860 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.989818    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:02.990014 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.989862    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3223957-508e-40d6-a6aa-6073e8438ca5-iptables-alerter-script\") pod \"iptables-alerter-nfqw9\" (UID: \"f3223957-508e-40d6-a6aa-6073e8438ca5\") " pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:02.990014 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.989890    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-host\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.990014 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.989910    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-hostroot\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.990014 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.989929    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-socket-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:02.990014 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.989960    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec-hosts-file\") pod \"node-resolver-twtj4\" (UID: \"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec\") " pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:02.990014 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.989999    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-var-lib-openvswitch\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990017    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-hostroot\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990025    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-run-openvswitch\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990057    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-host\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990062    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-run-openvswitch\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990068    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-run-ovn\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990114    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cn9g\" (UniqueName: \"kubernetes.io/projected/94021732-9afe-4fc7-aad6-b71ebbed7af9-kube-api-access-2cn9g\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-run-ovn\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990114    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-var-lib-openvswitch\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990145    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-tmp\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990178    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-etc-kubernetes\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990205    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd3fdc46-edd4-4833-baca-c730c52fed08-cnibin\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990232    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-run-ovn-kubernetes\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990336 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990330    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd3fdc46-edd4-4833-baca-c730c52fed08-cnibin\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990352    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-etc-kubernetes\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990382    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-run-ovn-kubernetes\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990418    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-lib-modules\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-run-multus-certs\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:02.990497    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990515    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-systemd-units\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990538    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94021732-9afe-4fc7-aad6-b71ebbed7af9-ovnkube-script-lib\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-sysconfig\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:02.990589    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs podName:4d739055-efd4-41fe-9725-197913a5257c nodeName:}" failed. No retries permitted until 2026-09-11 00:58:03.490564854 +0000 UTC m=+3.005165035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs") pod "network-metrics-daemon-7qkdk" (UID: "4d739055-efd4-41fe-9725-197913a5257c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990602    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-sysconfig\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990624    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-var-lib-kubelet\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990644    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3223957-508e-40d6-a6aa-6073e8438ca5-iptables-alerter-script\") pod \"iptables-alerter-nfqw9\" (UID: \"f3223957-508e-40d6-a6aa-6073e8438ca5\") " pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990697    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-run-multus-certs\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990697    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-lib-modules\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990735    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd3fdc46-edd4-4833-baca-c730c52fed08-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990726    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-systemd-units\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.990915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990768    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990795    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tfvf\" (UniqueName: \"kubernetes.io/projected/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-kube-api-access-2tfvf\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990823    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-run-netns\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990868    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-node-log\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990891    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-modprobe-d\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990928    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-sys\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990955    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-systemd\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.990980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-tuned\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991020    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-node-log\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991096    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-run-netns\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991087    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-cnibin\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991142    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-cni-bin\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991167    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-sysctl-conf\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991186    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991223    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-modprobe-d\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991259    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-var-lib-cni-bin\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991279    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-sys\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991328    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-systemd\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.991734 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991365    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-cni-bin\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.992593 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991381    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-cnibin\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.992593 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991393    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/94021732-9afe-4fc7-aad6-b71ebbed7af9-ovnkube-script-lib\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.992593 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991432    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-var-lib-kubelet\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.992593 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991214    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-var-lib-cni-bin\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.992593 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.992334    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-sysctl-conf\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.993015 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.992982    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl2tc\" (UniqueName: \"kubernetes.io/projected/dd3fdc46-edd4-4833-baca-c730c52fed08-kube-api-access-kl2tc\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.993084 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.993053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-registration-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:02.993135 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.993109    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/472cc964-fb4d-4f63-bd39-3fdbbcaf868f-konnectivity-ca\") pod \"konnectivity-agent-zndzf\" (UID: \"472cc964-fb4d-4f63-bd39-3fdbbcaf868f\") " pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:02.993188 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.993144    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-etc-openvswitch\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.993188 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.993171    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-kubernetes\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.993292 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.993204    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a670085-8845-40b7-ab78-2d028806840c-host\") pod \"node-ca-hrx2c\" (UID: \"0a670085-8845-40b7-ab78-2d028806840c\") " pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:02.993292 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.993249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcfnm\" (UniqueName: \"kubernetes.io/projected/820c3bfc-578f-41c3-97c1-5a5a356345a8-kube-api-access-rcfnm\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.993507 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.993486    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a670085-8845-40b7-ab78-2d028806840c-serviceca\") pod \"node-ca-hrx2c\" (UID: \"0a670085-8845-40b7-ab78-2d028806840c\") " pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:02.993566 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.993537    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-var-lib-kubelet\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.993625 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.993582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-daemon-config\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.993736 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.991508    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd3fdc46-edd4-4833-baca-c730c52fed08-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.994043 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a670085-8845-40b7-ab78-2d028806840c-host\") pod \"node-ca-hrx2c\" (UID: \"0a670085-8845-40b7-ab78-2d028806840c\") " pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:02.994114 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994100    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-var-lib-kubelet\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.994232 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994212    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a670085-8845-40b7-ab78-2d028806840c-serviceca\") pod \"node-ca-hrx2c\" (UID: \"0a670085-8845-40b7-ab78-2d028806840c\") " pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:02.994284 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.993709    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-kubernetes\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.994284 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994266    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd3fdc46-edd4-4833-baca-c730c52fed08-system-cni-dir\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.994396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994305    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-etc-openvswitch\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.994396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994314    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-sys-fs\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:02.994396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994346    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/472cc964-fb4d-4f63-bd39-3fdbbcaf868f-agent-certs\") pod \"konnectivity-agent-zndzf\" (UID: \"472cc964-fb4d-4f63-bd39-3fdbbcaf868f\") " pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:02.994396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994384    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrcwx\" (UniqueName: \"kubernetes.io/projected/0a670085-8845-40b7-ab78-2d028806840c-kube-api-access-nrcwx\") pod \"node-ca-hrx2c\" (UID: \"0a670085-8845-40b7-ab78-2d028806840c\") " pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:02.994396 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994387    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd3fdc46-edd4-4833-baca-c730c52fed08-system-cni-dir\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.994728 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-run-k8s-cni-cncf-io\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.994728 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994669    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-run-k8s-cni-cncf-io\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.994728 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994710    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-run-netns\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.994878 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd3fdc46-edd4-4833-baca-c730c52fed08-cni-binary-copy\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.994878 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994785    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:02.994878 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994816    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-log-socket\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.994878 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994831    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-run-netns\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.994878 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994843    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.994878 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994840    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-daemon-config\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.995166 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994883    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94021732-9afe-4fc7-aad6-b71ebbed7af9-ovn-node-metrics-cert\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.995166 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994915    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-sysctl-d\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.995166 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994921    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.995166 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994947    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-system-cni-dir\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.995166 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994914    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-log-socket\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.995166 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.994974    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-socket-dir-parent\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.995166 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995148    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-system-cni-dir\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.995473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-conf-dir\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.995473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995217    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlw2b\" (UniqueName: \"kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b\") pod \"network-check-target-v55tj\" (UID: \"39e30de3-b555-45b2-84b5-0c93e875bddb\") " pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:02.995473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995229    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-socket-dir-parent\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.995473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995256    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec-tmp-dir\") pod \"node-resolver-twtj4\" (UID: \"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec\") " pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:02.995473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995290    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94021732-9afe-4fc7-aad6-b71ebbed7af9-ovnkube-config\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.995473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995317    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvpv9\" (UniqueName: \"kubernetes.io/projected/f3223957-508e-40d6-a6aa-6073e8438ca5-kube-api-access-lvpv9\") pod \"iptables-alerter-nfqw9\" (UID: \"f3223957-508e-40d6-a6aa-6073e8438ca5\") " pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:02.995473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/820c3bfc-578f-41c3-97c1-5a5a356345a8-cni-binary-copy\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.995473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995383    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd3fdc46-edd4-4833-baca-c730c52fed08-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.995473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995415    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-conf-dir\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.995473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxcp4\" (UniqueName: \"kubernetes.io/projected/4d739055-efd4-41fe-9725-197913a5257c-kube-api-access-bxcp4\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:02.995885 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995589    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94021732-9afe-4fc7-aad6-b71ebbed7af9-env-overrides\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.995885 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995620    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-var-lib-cni-multus\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.995885 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995633    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-tuned\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.995885 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995650    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd3fdc46-edd4-4833-baca-c730c52fed08-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.995885 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995702    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2484980c-48af-426f-b3c2-95afc18ed2a2-kubelet-config\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:02.995885 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995733    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2484980c-48af-426f-b3c2-95afc18ed2a2-dbus\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:02.995885 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995762    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-etc-selinux\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:02.995885 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995802    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-kubelet\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.995885 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995833    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-slash\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.995885 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995864    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-run-systemd\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.996242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995897    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-run\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.996242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-device-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:02.996242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996000    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-cni-netd\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.996242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996134    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-cni-netd\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.996589 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996566    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd3fdc46-edd4-4833-baca-c730c52fed08-cni-binary-copy\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.996664 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996631    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/820c3bfc-578f-41c3-97c1-5a5a356345a8-cni-binary-copy\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.996745 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996666    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-host-var-lib-cni-multus\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.996745 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.995219    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-etc-sysctl-d\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.996860 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996761    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-run-systemd\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.996860 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996817    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-kubelet\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.996860 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996840    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd3fdc46-edd4-4833-baca-c730c52fed08-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.997015 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996867    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94021732-9afe-4fc7-aad6-b71ebbed7af9-host-slash\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.997015 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996896    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd3fdc46-edd4-4833-baca-c730c52fed08-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:02.997015 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996909    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3223957-508e-40d6-a6aa-6073e8438ca5-host-slash\") pod \"iptables-alerter-nfqw9\" (UID: \"f3223957-508e-40d6-a6aa-6073e8438ca5\") " pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:02.997015 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996912    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2484980c-48af-426f-b3c2-95afc18ed2a2-kubelet-config\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:02.997015 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvqdn\" (UniqueName: \"kubernetes.io/projected/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-kube-api-access-dvqdn\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.997015 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996979    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3223957-508e-40d6-a6aa-6073e8438ca5-host-slash\") pod \"iptables-alerter-nfqw9\" (UID: \"f3223957-508e-40d6-a6aa-6073e8438ca5\") " pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:02.997015 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.996995    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-cni-dir\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.997366 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.997036    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-run\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:02.997366 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.997034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdrs6\" (UniqueName: \"kubernetes.io/projected/a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec-kube-api-access-kdrs6\") pod \"node-resolver-twtj4\" (UID: \"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec\") " pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:02.997366 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.997117    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2484980c-48af-426f-b3c2-95afc18ed2a2-dbus\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:02.997366 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.997138    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/94021732-9afe-4fc7-aad6-b71ebbed7af9-ovnkube-config\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.997366 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.997214    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/820c3bfc-578f-41c3-97c1-5a5a356345a8-multus-cni-dir\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:02.997783 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.997763    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/94021732-9afe-4fc7-aad6-b71ebbed7af9-ovn-node-metrics-cert\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.997936 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:02.997913    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:02.998002 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:02.997992    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret podName:2484980c-48af-426f-b3c2-95afc18ed2a2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:03.497973973 +0000 UTC m=+3.012574138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret") pod "global-pull-secret-syncer-gg5rc" (UID: "2484980c-48af-426f-b3c2-95afc18ed2a2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:02.998168 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.998138    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/94021732-9afe-4fc7-aad6-b71ebbed7af9-env-overrides\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:02.999077 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:02.999054    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-tmp\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:03.003028 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.002966    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cn9g\" (UniqueName: \"kubernetes.io/projected/94021732-9afe-4fc7-aad6-b71ebbed7af9-kube-api-access-2cn9g\") pod \"ovnkube-node-lr2p4\" (UID: \"94021732-9afe-4fc7-aad6-b71ebbed7af9\") " pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:03.004929 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.004900    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcfnm\" (UniqueName: \"kubernetes.io/projected/820c3bfc-578f-41c3-97c1-5a5a356345a8-kube-api-access-rcfnm\") pod \"multus-52jsv\" (UID: \"820c3bfc-578f-41c3-97c1-5a5a356345a8\") " pod="openshift-multus/multus-52jsv"
Sep 11 00:58:03.005046 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.005025    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:03.005122 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.005055    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:03.005122 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.005071    2559 projected.go:196] Error preparing data for projected volume kube-api-access-dlw2b for pod openshift-network-diagnostics/network-check-target-v55tj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:03.005232 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.005146    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b podName:39e30de3-b555-45b2-84b5-0c93e875bddb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:03.50512389 +0000 UTC m=+3.019724074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-dlw2b" (UniqueName: "kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b") pod "network-check-target-v55tj" (UID: "39e30de3-b555-45b2-84b5-0c93e875bddb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:03.006961 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.006933    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvqdn\" (UniqueName: \"kubernetes.io/projected/d6681fa8-d9d9-4fce-b2cd-6d852ec98216-kube-api-access-dvqdn\") pod \"tuned-rf5j7\" (UID: \"d6681fa8-d9d9-4fce-b2cd-6d852ec98216\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:03.007347 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.007314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvpv9\" (UniqueName: \"kubernetes.io/projected/f3223957-508e-40d6-a6aa-6073e8438ca5-kube-api-access-lvpv9\") pod \"iptables-alerter-nfqw9\" (UID: \"f3223957-508e-40d6-a6aa-6073e8438ca5\") " pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:03.007428 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.007405    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl2tc\" (UniqueName: \"kubernetes.io/projected/dd3fdc46-edd4-4833-baca-c730c52fed08-kube-api-access-kl2tc\") pod \"multus-additional-cni-plugins-tgf62\" (UID: \"dd3fdc46-edd4-4833-baca-c730c52fed08\") " pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:03.007474 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.007453    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrcwx\" (UniqueName: \"kubernetes.io/projected/0a670085-8845-40b7-ab78-2d028806840c-kube-api-access-nrcwx\") pod \"node-ca-hrx2c\" (UID: \"0a670085-8845-40b7-ab78-2d028806840c\") " pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:03.007741 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.007724    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxcp4\" (UniqueName: \"kubernetes.io/projected/4d739055-efd4-41fe-9725-197913a5257c-kube-api-access-bxcp4\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:03.064205 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.064166    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 00:58:03.098561 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098521    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec-tmp-dir\") pod \"node-resolver-twtj4\" (UID: \"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec\") " pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:03.098765 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-etc-selinux\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.098765 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098604    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-device-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.098765 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098644    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdrs6\" (UniqueName: \"kubernetes.io/projected/a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec-kube-api-access-kdrs6\") pod \"node-resolver-twtj4\" (UID: \"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec\") " pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:03.098765 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098734    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-device-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099006 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098787    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-socket-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099006 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098818    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec-hosts-file\") pod \"node-resolver-twtj4\" (UID: \"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec\") " pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:03.099006 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098784    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-etc-selinux\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099006 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098861    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099006 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098891    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tfvf\" (UniqueName: \"kubernetes.io/projected/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-kube-api-access-2tfvf\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099006 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec-tmp-dir\") pod \"node-resolver-twtj4\" (UID: \"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec\") " pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:03.099006 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.098993    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.099011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-registration-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.099038    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/472cc964-fb4d-4f63-bd39-3fdbbcaf868f-konnectivity-ca\") pod \"konnectivity-agent-zndzf\" (UID: \"472cc964-fb4d-4f63-bd39-3fdbbcaf868f\") " pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:03.099242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.099048    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec-hosts-file\") pod \"node-resolver-twtj4\" (UID: \"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec\") " pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:03.099242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.099065    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-socket-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.099081    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-sys-fs\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.099132    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-registration-dir\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.099137    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-sys-fs\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.099242 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.099165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/472cc964-fb4d-4f63-bd39-3fdbbcaf868f-agent-certs\") pod \"konnectivity-agent-zndzf\" (UID: \"472cc964-fb4d-4f63-bd39-3fdbbcaf868f\") " pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:03.099531 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.099514    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/472cc964-fb4d-4f63-bd39-3fdbbcaf868f-konnectivity-ca\") pod \"konnectivity-agent-zndzf\" (UID: \"472cc964-fb4d-4f63-bd39-3fdbbcaf868f\") " pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:03.102362 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.102341    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/472cc964-fb4d-4f63-bd39-3fdbbcaf868f-agent-certs\") pod \"konnectivity-agent-zndzf\" (UID: \"472cc964-fb4d-4f63-bd39-3fdbbcaf868f\") " pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:03.108509 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.108482    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdrs6\" (UniqueName: \"kubernetes.io/projected/a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec-kube-api-access-kdrs6\") pod \"node-resolver-twtj4\" (UID: \"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec\") " pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:03.108633 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.108484    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tfvf\" (UniqueName: \"kubernetes.io/projected/17f626d1-dfe3-4af5-9ea8-ce84697a9b51-kube-api-access-2tfvf\") pod \"aws-ebs-csi-driver-node-qw9w8\" (UID: \"17f626d1-dfe3-4af5-9ea8-ce84697a9b51\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.109754 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.109697    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal" event={"ID":"e42a195d24144df11416cc5ea52d2a36","Type":"ContainerStarted","Data":"9d9afb91839d458ecb0ae0cf09a0e530ee40566c9941217c3a4e20adf2b7cc9d"}
Sep 11 00:58:03.111312 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.111274    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal" event={"ID":"631b5c907d6ae04b2ef2175a9c73889c","Type":"ContainerStarted","Data":"44d4bbc6078947e219dd10ee41c14477708cd85b718cae5c599a21d078f93e91"}
Sep 11 00:58:03.188208 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.188090    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nfqw9"
Sep 11 00:58:03.202015 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.201892    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rf5j7"
Sep 11 00:58:03.209048 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.209022    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-52jsv"
Sep 11 00:58:03.218449 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.218419    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tgf62"
Sep 11 00:58:03.226517 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.226488    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hrx2c"
Sep 11 00:58:03.234733 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.234713    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:03.242751 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.242727    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8"
Sep 11 00:58:03.251817 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.251794    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:03.258915 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.258894    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-twtj4"
Sep 11 00:58:03.502531 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.502456    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:03.502531 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.502510    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:03.502716 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.502618    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:03.502716 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.502634    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:03.502716 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.502687    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs podName:4d739055-efd4-41fe-9725-197913a5257c nodeName:}" failed. No retries permitted until 2026-09-11 00:58:04.502671017 +0000 UTC m=+4.017271179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs") pod "network-metrics-daemon-7qkdk" (UID: "4d739055-efd4-41fe-9725-197913a5257c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:03.502716 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.502702    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret podName:2484980c-48af-426f-b3c2-95afc18ed2a2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:04.502694149 +0000 UTC m=+4.017294312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret") pod "global-pull-secret-syncer-gg5rc" (UID: "2484980c-48af-426f-b3c2-95afc18ed2a2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:03.603416 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.603378    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlw2b\" (UniqueName: \"kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b\") pod \"network-check-target-v55tj\" (UID: \"39e30de3-b555-45b2-84b5-0c93e875bddb\") " pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:03.603607 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.603524    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:03.603607 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.603543    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:03.603607 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.603552    2559 projected.go:196] Error preparing data for projected volume kube-api-access-dlw2b for pod openshift-network-diagnostics/network-check-target-v55tj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:03.603810 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:03.603615    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b podName:39e30de3-b555-45b2-84b5-0c93e875bddb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:04.603595721 +0000 UTC m=+4.118195908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlw2b" (UniqueName: "kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b") pod "network-check-target-v55tj" (UID: "39e30de3-b555-45b2-84b5-0c93e875bddb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:03.921972 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.921878    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 00:53:03 +0000 UTC" deadline="2028-03-19 16:15:42.123857971 +0000 UTC"
Sep 11 00:58:03.921972 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:03.921919    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13335h17m38.201944166s"
Sep 11 00:58:04.103749 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:04.103710    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:04.103956 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:04.103763    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:04.103956 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:04.103856    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:04.104074 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:04.104006    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:04.214029 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:04.214000    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod820c3bfc_578f_41c3_97c1_5a5a356345a8.slice/crio-2a39170db03e09322e882d47e6c1c5901040df1f9b8724aa9bd91472ee382e42 WatchSource:0}: Error finding container 2a39170db03e09322e882d47e6c1c5901040df1f9b8724aa9bd91472ee382e42: Status 404 returned error can't find the container with id 2a39170db03e09322e882d47e6c1c5901040df1f9b8724aa9bd91472ee382e42
Sep 11 00:58:04.218867 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:04.218254    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2aee08b_0ce8_4ef5_87f8_c3f06d9f97ec.slice/crio-58b0cd6071185bc8de1427546dde99139f50172fc2c35ff9d37fda3cfdfa649e WatchSource:0}: Error finding container 58b0cd6071185bc8de1427546dde99139f50172fc2c35ff9d37fda3cfdfa649e: Status 404 returned error can't find the container with id 58b0cd6071185bc8de1427546dde99139f50172fc2c35ff9d37fda3cfdfa649e
Sep 11 00:58:04.220456 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:04.219657    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3223957_508e_40d6_a6aa_6073e8438ca5.slice/crio-5d8660b11b41f92701d8b35b3eafe0ae0a3ea478c86178f62335c26013fdc93e WatchSource:0}: Error finding container 5d8660b11b41f92701d8b35b3eafe0ae0a3ea478c86178f62335c26013fdc93e: Status 404 returned error can't find the container with id 5d8660b11b41f92701d8b35b3eafe0ae0a3ea478c86178f62335c26013fdc93e
Sep 11 00:58:04.220746 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:04.220675    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472cc964_fb4d_4f63_bd39_3fdbbcaf868f.slice/crio-3f9e197cbf4affaa7c3e9fe15e881dc3d170fb7d90e1d891915dc946581ef0bb WatchSource:0}: Error finding container 3f9e197cbf4affaa7c3e9fe15e881dc3d170fb7d90e1d891915dc946581ef0bb: Status 404 returned error can't find the container with id 3f9e197cbf4affaa7c3e9fe15e881dc3d170fb7d90e1d891915dc946581ef0bb
Sep 11 00:58:04.228724 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:04.228296    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd3fdc46_edd4_4833_baca_c730c52fed08.slice/crio-c10d20848e926ac5fa306498481199e23b352891c9587e9487cf054726e31404 WatchSource:0}: Error finding container c10d20848e926ac5fa306498481199e23b352891c9587e9487cf054726e31404: Status 404 returned error can't find the container with id c10d20848e926ac5fa306498481199e23b352891c9587e9487cf054726e31404
Sep 11 00:58:04.230057 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:04.229628    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6681fa8_d9d9_4fce_b2cd_6d852ec98216.slice/crio-77f09f6aed3d502ac54b141ef6918a4602c3523cb11056a7340c072f15bee990 WatchSource:0}: Error finding container 77f09f6aed3d502ac54b141ef6918a4602c3523cb11056a7340c072f15bee990: Status 404 returned error can't find the container with id 77f09f6aed3d502ac54b141ef6918a4602c3523cb11056a7340c072f15bee990
Sep 11 00:58:04.231498 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:04.231124    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94021732_9afe_4fc7_aad6_b71ebbed7af9.slice/crio-60d6612ef50a6979ece1b599a626a43a7d81f12181e2f33767b7f1ed79b5318b WatchSource:0}: Error finding container 60d6612ef50a6979ece1b599a626a43a7d81f12181e2f33767b7f1ed79b5318b: Status 404 returned error can't find the container with id 60d6612ef50a6979ece1b599a626a43a7d81f12181e2f33767b7f1ed79b5318b
Sep 11 00:58:04.512099 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:04.512046    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:04.512241 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:04.512118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:04.512241 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:04.512224    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:04.512358 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:04.512307    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret podName:2484980c-48af-426f-b3c2-95afc18ed2a2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:06.512282581 +0000 UTC m=+6.026882765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret") pod "global-pull-secret-syncer-gg5rc" (UID: "2484980c-48af-426f-b3c2-95afc18ed2a2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:04.512358 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:04.512230    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:04.512478 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:04.512382    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs podName:4d739055-efd4-41fe-9725-197913a5257c nodeName:}" failed. No retries permitted until 2026-09-11 00:58:06.512367264 +0000 UTC m=+6.026967429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs") pod "network-metrics-daemon-7qkdk" (UID: "4d739055-efd4-41fe-9725-197913a5257c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:04.613309 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:04.613271    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlw2b\" (UniqueName: \"kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b\") pod \"network-check-target-v55tj\" (UID: \"39e30de3-b555-45b2-84b5-0c93e875bddb\") " pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:04.613507 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:04.613464    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:04.613507 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:04.613491    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:04.613507 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:04.613505    2559 projected.go:196] Error preparing data for projected volume kube-api-access-dlw2b for pod openshift-network-diagnostics/network-check-target-v55tj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:04.613700 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:04.613580    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b podName:39e30de3-b555-45b2-84b5-0c93e875bddb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:06.613558542 +0000 UTC m=+6.128158722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlw2b" (UniqueName: "kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b") pod "network-check-target-v55tj" (UID: "39e30de3-b555-45b2-84b5-0c93e875bddb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:04.926671 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:04.922067    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 00:53:03 +0000 UTC" deadline="2028-05-22 21:19:08.629886314 +0000 UTC"
Sep 11 00:58:04.926671 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:04.922134    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14876h21m3.707757152s"
Sep 11 00:58:05.104908 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.104048    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:05.104908 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:05.104220    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:05.129699 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.125708    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rf5j7" event={"ID":"d6681fa8-d9d9-4fce-b2cd-6d852ec98216","Type":"ContainerStarted","Data":"77f09f6aed3d502ac54b141ef6918a4602c3523cb11056a7340c072f15bee990"}
Sep 11 00:58:05.134241 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.134187    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tgf62" event={"ID":"dd3fdc46-edd4-4833-baca-c730c52fed08","Type":"ContainerStarted","Data":"c10d20848e926ac5fa306498481199e23b352891c9587e9487cf054726e31404"}
Sep 11 00:58:05.148234 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.148193    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hrx2c" event={"ID":"0a670085-8845-40b7-ab78-2d028806840c","Type":"ContainerStarted","Data":"0a36b6bc8ecbb23341b102ab0b0629271c5d772e7417339412268a9a8dd29b4b"}
Sep 11 00:58:05.154877 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.154802    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-52jsv" event={"ID":"820c3bfc-578f-41c3-97c1-5a5a356345a8","Type":"ContainerStarted","Data":"2a39170db03e09322e882d47e6c1c5901040df1f9b8724aa9bd91472ee382e42"}
Sep 11 00:58:05.170562 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.170526    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal" event={"ID":"e42a195d24144df11416cc5ea52d2a36","Type":"ContainerStarted","Data":"fc2a2294d17d6ee3010ed4690af043881fdf997f18c65ebb590d7c928761535a"}
Sep 11 00:58:05.174667 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.174633    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8" event={"ID":"17f626d1-dfe3-4af5-9ea8-ce84697a9b51","Type":"ContainerStarted","Data":"502db96fb3bd9c3bb16ee2d1be94a7b945f038a04ff5cdef20b4306ea30bd64d"}
Sep 11 00:58:05.181374 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.181249    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zndzf" event={"ID":"472cc964-fb4d-4f63-bd39-3fdbbcaf868f","Type":"ContainerStarted","Data":"3f9e197cbf4affaa7c3e9fe15e881dc3d170fb7d90e1d891915dc946581ef0bb"}
Sep 11 00:58:05.196691 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.196421    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nfqw9" event={"ID":"f3223957-508e-40d6-a6aa-6073e8438ca5","Type":"ContainerStarted","Data":"5d8660b11b41f92701d8b35b3eafe0ae0a3ea478c86178f62335c26013fdc93e"}
Sep 11 00:58:05.209178 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.209137    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-twtj4" event={"ID":"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec","Type":"ContainerStarted","Data":"58b0cd6071185bc8de1427546dde99139f50172fc2c35ff9d37fda3cfdfa649e"}
Sep 11 00:58:05.213599 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.213566    2559 generic.go:356] "Generic (PLEG): container finished" podID="631b5c907d6ae04b2ef2175a9c73889c" containerID="957e1034873eb94c9717463bcf17edfcdbf8e06e18fe1c137962f43683315f26" exitCode=0
Sep 11 00:58:05.213744 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.213673    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal" event={"ID":"631b5c907d6ae04b2ef2175a9c73889c","Type":"ContainerDied","Data":"957e1034873eb94c9717463bcf17edfcdbf8e06e18fe1c137962f43683315f26"}
Sep 11 00:58:05.216657 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.216630    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4" event={"ID":"94021732-9afe-4fc7-aad6-b71ebbed7af9","Type":"ContainerStarted","Data":"60d6612ef50a6979ece1b599a626a43a7d81f12181e2f33767b7f1ed79b5318b"}
Sep 11 00:58:05.232783 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:05.232338    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-14.ec2.internal" podStartSLOduration=3.232315379 podStartE2EDuration="3.232315379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:05.189785482 +0000 UTC m=+4.704385711" watchObservedRunningTime="2026-09-11 00:58:05.232315379 +0000 UTC m=+4.746915574"
Sep 11 00:58:06.104576 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:06.104539    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:06.105327 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:06.104700    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:06.105327 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:06.105135    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:06.105327 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:06.105246    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:06.244168 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:06.244132    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal" event={"ID":"631b5c907d6ae04b2ef2175a9c73889c","Type":"ContainerStarted","Data":"20ae38e3c8eb6682a41674386f14961b0fe6f576d8999ceefbab716f71b2c783"}
Sep 11 00:58:06.261495 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:06.261415    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-14.ec2.internal" podStartSLOduration=4.26139213 podStartE2EDuration="4.26139213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:06.260667811 +0000 UTC m=+5.775268008" watchObservedRunningTime="2026-09-11 00:58:06.26139213 +0000 UTC m=+5.775992314"
Sep 11 00:58:06.529528 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:06.529408    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:06.529528 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:06.529491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:06.529756 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:06.529625    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:06.529756 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:06.529696    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs podName:4d739055-efd4-41fe-9725-197913a5257c nodeName:}" failed. No retries permitted until 2026-09-11 00:58:10.529666588 +0000 UTC m=+10.044266752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs") pod "network-metrics-daemon-7qkdk" (UID: "4d739055-efd4-41fe-9725-197913a5257c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:06.530245 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:06.530143    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:06.530245 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:06.530205    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret podName:2484980c-48af-426f-b3c2-95afc18ed2a2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:10.530188003 +0000 UTC m=+10.044788168 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret") pod "global-pull-secret-syncer-gg5rc" (UID: "2484980c-48af-426f-b3c2-95afc18ed2a2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:06.630886 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:06.630311    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlw2b\" (UniqueName: \"kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b\") pod \"network-check-target-v55tj\" (UID: \"39e30de3-b555-45b2-84b5-0c93e875bddb\") " pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:06.630886 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:06.630483    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:06.630886 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:06.630499    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:06.630886 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:06.630508    2559 projected.go:196] Error preparing data for projected volume kube-api-access-dlw2b for pod openshift-network-diagnostics/network-check-target-v55tj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:06.630886 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:06.630554    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b podName:39e30de3-b555-45b2-84b5-0c93e875bddb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:10.630542133 +0000 UTC m=+10.145142309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlw2b" (UniqueName: "kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b") pod "network-check-target-v55tj" (UID: "39e30de3-b555-45b2-84b5-0c93e875bddb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:07.107051 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:07.106456    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:07.107051 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:07.106600    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:08.104767 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:08.104279    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:08.104767 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:08.104407    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:08.104767 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:08.104451    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:08.104767 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:08.104562    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:09.106872 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:09.106350    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:09.106872 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:09.106491    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:10.104375 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:10.104338    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:10.104545 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:10.104479    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:10.104642 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:10.104566    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:10.104707 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:10.104665    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:10.569654 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:10.569512    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:10.569654 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:10.569584    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:10.570145 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:10.569704    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:10.570145 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:10.569723    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:10.570145 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:10.569784    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret podName:2484980c-48af-426f-b3c2-95afc18ed2a2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:18.569764535 +0000 UTC m=+18.084364710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret") pod "global-pull-secret-syncer-gg5rc" (UID: "2484980c-48af-426f-b3c2-95afc18ed2a2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:10.570145 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:10.569805    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs podName:4d739055-efd4-41fe-9725-197913a5257c nodeName:}" failed. No retries permitted until 2026-09-11 00:58:18.569794384 +0000 UTC m=+18.084394544 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs") pod "network-metrics-daemon-7qkdk" (UID: "4d739055-efd4-41fe-9725-197913a5257c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:10.671272 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:10.670681    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlw2b\" (UniqueName: \"kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b\") pod \"network-check-target-v55tj\" (UID: \"39e30de3-b555-45b2-84b5-0c93e875bddb\") " pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:10.671272 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:10.670872    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:10.671272 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:10.670892    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:10.671272 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:10.670906    2559 projected.go:196] Error preparing data for projected volume kube-api-access-dlw2b for pod openshift-network-diagnostics/network-check-target-v55tj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:10.671272 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:10.670966    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b podName:39e30de3-b555-45b2-84b5-0c93e875bddb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:18.670948367 +0000 UTC m=+18.185548545 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlw2b" (UniqueName: "kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b") pod "network-check-target-v55tj" (UID: "39e30de3-b555-45b2-84b5-0c93e875bddb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:11.104760 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:11.104555    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:11.104938 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:11.104854    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:12.104071 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:12.104037    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:12.104528 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:12.104170    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:12.104528 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:12.104230    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:12.104528 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:12.104305    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:13.104353 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:13.104313    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:13.104815 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:13.104507    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:14.104080 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:14.104039    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:14.104239 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:14.104039    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:14.104239 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:14.104164    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:14.104239 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:14.104209    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:15.104549 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:15.104510    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:15.104974 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:15.104671    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:16.103655 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:16.103626    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:16.103840 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:16.103721    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:16.103840 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:16.103784    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:16.103945 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:16.103897    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:17.103804 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:17.103760    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:17.104227 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:17.103920    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:18.103991 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:18.103953    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:18.103991 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:18.103975    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:18.104515 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:18.104089    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:18.104515 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:18.104219    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:18.630598 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:18.630532    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:18.630761 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:18.630647    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:18.630761 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:18.630710    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:18.630761 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:18.630748    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:18.630979 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:18.630776    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs podName:4d739055-efd4-41fe-9725-197913a5257c nodeName:}" failed. No retries permitted until 2026-09-11 00:58:34.630761396 +0000 UTC m=+34.145361572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs") pod "network-metrics-daemon-7qkdk" (UID: "4d739055-efd4-41fe-9725-197913a5257c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:18.630979 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:18.630800    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret podName:2484980c-48af-426f-b3c2-95afc18ed2a2 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:34.630783799 +0000 UTC m=+34.145383974 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret") pod "global-pull-secret-syncer-gg5rc" (UID: "2484980c-48af-426f-b3c2-95afc18ed2a2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:18.732012 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:18.731971    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlw2b\" (UniqueName: \"kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b\") pod \"network-check-target-v55tj\" (UID: \"39e30de3-b555-45b2-84b5-0c93e875bddb\") " pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:18.732160 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:18.732142    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:18.732213 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:18.732166    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:18.732213 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:18.732179    2559 projected.go:196] Error preparing data for projected volume kube-api-access-dlw2b for pod openshift-network-diagnostics/network-check-target-v55tj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:18.732292 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:18.732235    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b podName:39e30de3-b555-45b2-84b5-0c93e875bddb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:34.73222043 +0000 UTC m=+34.246820722 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlw2b" (UniqueName: "kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b") pod "network-check-target-v55tj" (UID: "39e30de3-b555-45b2-84b5-0c93e875bddb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:19.104282 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:19.104246    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:19.104734 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:19.104377    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:20.104334 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:20.104295    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:20.104771 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:20.104295    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:20.104771 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:20.104445    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:20.104771 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:20.104481    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:21.104947 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:21.104917    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:21.105337 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:21.105015    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:22.103926 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:22.103888    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:22.104111 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:22.103890    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:22.104111 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:22.104045    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:22.104241 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:22.104140    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:23.104065 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.104031    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:23.104703 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:23.104181    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:23.281882 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.281850    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8" event={"ID":"17f626d1-dfe3-4af5-9ea8-ce84697a9b51","Type":"ContainerStarted","Data":"8ba3c8f0fd5539103bd2874d0b39f0636171bf9e0ee9fbbf5341a87592cd963a"}
Sep 11 00:58:23.283243 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.283218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zndzf" event={"ID":"472cc964-fb4d-4f63-bd39-3fdbbcaf868f","Type":"ContainerStarted","Data":"4ab6486180b26fd97f5255d35cbf9747b137fa787de6e9436e5ec32f2a8b1801"}
Sep 11 00:58:23.284656 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.284636    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-twtj4" event={"ID":"a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec","Type":"ContainerStarted","Data":"48e26ccf230991407db62b8accf628f06d28b057baad0ca7e67fefcf3c0b255c"}
Sep 11 00:58:23.287652 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.287632    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4" event={"ID":"94021732-9afe-4fc7-aad6-b71ebbed7af9","Type":"ContainerStarted","Data":"2fb3aeeab488daf1219e94101a11b9b03d1a49f8f3d74bde8300ea576d1b8594"}
Sep 11 00:58:23.287744 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.287707    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4" event={"ID":"94021732-9afe-4fc7-aad6-b71ebbed7af9","Type":"ContainerStarted","Data":"9c479754797c4a83022afb0e0f4ce405e95c5d9dc36b2c1ce4e69f5799069e49"}
Sep 11 00:58:23.287786 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.287758    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4" event={"ID":"94021732-9afe-4fc7-aad6-b71ebbed7af9","Type":"ContainerStarted","Data":"f1d5324ba590ccb4e2987338b7ebc0606b7510086099069eb2c7c6809d883e3b"}
Sep 11 00:58:23.287826 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.287807    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4" event={"ID":"94021732-9afe-4fc7-aad6-b71ebbed7af9","Type":"ContainerStarted","Data":"6f74191286a5cbe4ba5368d3fc390a3f201138887dacf6e72a3ffa2bd8f3db3f"}
Sep 11 00:58:23.287864 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.287850    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4" event={"ID":"94021732-9afe-4fc7-aad6-b71ebbed7af9","Type":"ContainerStarted","Data":"b67a7cad2a24091ade8fcb75e95063ada5067996b2cb1df3c0f508eafa228ac4"}
Sep 11 00:58:23.287952 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.287881    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4" event={"ID":"94021732-9afe-4fc7-aad6-b71ebbed7af9","Type":"ContainerStarted","Data":"ac54f1babd5281867edb328ab8130194199d52b381080248a9f516d512dd4979"}
Sep 11 00:58:23.289032 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.289007    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rf5j7" event={"ID":"d6681fa8-d9d9-4fce-b2cd-6d852ec98216","Type":"ContainerStarted","Data":"66fc56a5dd7a7515f59afdb80bde3280b7fb788535f34f197987614cc2309aae"}
Sep 11 00:58:23.290420 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.290400    2559 generic.go:356] "Generic (PLEG): container finished" podID="dd3fdc46-edd4-4833-baca-c730c52fed08" containerID="87acb30991f011ffd5c889f78d3e89c1a2802746c0961c40d909650d740adaa7" exitCode=0
Sep 11 00:58:23.290503 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.290459    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tgf62" event={"ID":"dd3fdc46-edd4-4833-baca-c730c52fed08","Type":"ContainerDied","Data":"87acb30991f011ffd5c889f78d3e89c1a2802746c0961c40d909650d740adaa7"}
Sep 11 00:58:23.292080 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.292061    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hrx2c" event={"ID":"0a670085-8845-40b7-ab78-2d028806840c","Type":"ContainerStarted","Data":"cd975d29cd796f0c300a540e2ef05abec8b56d902bd690b49f2d55b34a736e3e"}
Sep 11 00:58:23.293625 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.293593    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-52jsv" event={"ID":"820c3bfc-578f-41c3-97c1-5a5a356345a8","Type":"ContainerStarted","Data":"551c774e4df776ae6c78923cfcc89393db70de01286a9226904b58db3f87c435"}
Sep 11 00:58:23.317455 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.317404    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zndzf" podStartSLOduration=4.361203324 podStartE2EDuration="22.317384444s" totalImagesPullingTime="17.95618112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:23.297926395 +0000 UTC m=+22.812526577" watchObservedRunningTime="2026-09-11 00:58:23.317384444 +0000 UTC m=+22.831984621"
Sep 11 00:58:23.330491 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.330441    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-twtj4" podStartSLOduration=4.288224052 podStartE2EDuration="22.330428507s" totalImagesPullingTime="18.042204455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:23.330126072 +0000 UTC m=+22.844726254" watchObservedRunningTime="2026-09-11 00:58:23.330428507 +0000 UTC m=+22.845028690"
Sep 11 00:58:23.367214 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.367168    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hrx2c" podStartSLOduration=4.333667474 podStartE2EDuration="22.367149573s" totalImagesPullingTime="18.033482099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:23.350219423 +0000 UTC m=+22.864819606" watchObservedRunningTime="2026-09-11 00:58:23.367149573 +0000 UTC m=+22.881749808"
Sep 11 00:58:23.367354 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.367243    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-52jsv" podStartSLOduration=3.961698249 podStartE2EDuration="22.367238142s" totalImagesPullingTime="18.405539893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:23.366733373 +0000 UTC m=+22.881333554" watchObservedRunningTime="2026-09-11 00:58:23.367238142 +0000 UTC m=+22.881838325"
Sep 11 00:58:23.383534 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.383486    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rf5j7" podStartSLOduration=4.353894815 podStartE2EDuration="22.383470736s" totalImagesPullingTime="18.029575921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:23.383262496 +0000 UTC m=+22.897862678" watchObservedRunningTime="2026-09-11 00:58:23.383470736 +0000 UTC m=+22.898070928"
Sep 11 00:58:23.692266 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:23.692058    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 00:58:24.095839 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:24.095703    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T00:58:23.692262729Z","UUID":"8b27a0f2-7fb6-405d-be44-fababdd58f2b","Handler":null,"Name":"","Endpoint":""}
Sep 11 00:58:24.097526 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:24.097499    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 00:58:24.097675 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:24.097540    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 00:58:24.103718 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:24.103694    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:24.103843 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:24.103775    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:24.103843 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:24.103829    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:24.103961 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:24.103936    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:24.298502 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:24.298420    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8" event={"ID":"17f626d1-dfe3-4af5-9ea8-ce84697a9b51","Type":"ContainerStarted","Data":"0d73709df2d1641de6c91ea33161f05914e022ea3ca9befe68fe2295325dc5a0"}
Sep 11 00:58:24.301662 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:24.301281    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nfqw9" event={"ID":"f3223957-508e-40d6-a6aa-6073e8438ca5","Type":"ContainerStarted","Data":"7c7070a8f24d6512727d95d1969c838b6e9ed36e03b2fb153215e05bd9ef69ac"}
Sep 11 00:58:24.333451 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:24.333393    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nfqw9" podStartSLOduration=5.286992675 podStartE2EDuration="23.333372441s" totalImagesPullingTime="18.046379766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:24.332974592 +0000 UTC m=+23.847574775" watchObservedRunningTime="2026-09-11 00:58:24.333372441 +0000 UTC m=+23.847972638"
Sep 11 00:58:25.104619 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:25.104524    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:25.104783 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:25.104698    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:25.309181 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:25.309140    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4" event={"ID":"94021732-9afe-4fc7-aad6-b71ebbed7af9","Type":"ContainerStarted","Data":"1be5e77aa810b1c1719330873851444979f312be50a25a779493ff9d3d532d0b"}
Sep 11 00:58:25.311452 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:25.311416    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8" event={"ID":"17f626d1-dfe3-4af5-9ea8-ce84697a9b51","Type":"ContainerStarted","Data":"6d7bdbf42320d7030a11c00e3cdf9d23727701fb0ae67a5283fc0fd6bee3a2f3"}
Sep 11 00:58:25.334094 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:25.334044    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qw9w8" podStartSLOduration=4.343951277 podStartE2EDuration="24.334028049s" totalImagesPullingTime="19.990076772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:25.333783977 +0000 UTC m=+24.848384162" watchObservedRunningTime="2026-09-11 00:58:25.334028049 +0000 UTC m=+24.848628230"
Sep 11 00:58:26.103730 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:26.103695    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:26.103924 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:26.103694    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:26.103924 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:26.103848    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:26.104053 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:26.103926    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:26.881476 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:26.881440    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:26.882102 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:26.882081    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:27.103742 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:27.103709    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:27.103927 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:27.103872    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:27.315900 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:27.315862    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:27.316516 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:27.316499    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zndzf"
Sep 11 00:58:28.103695 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:28.103668    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:28.104315 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:28.103668    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:28.104315 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:28.103791    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:28.104315 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:28.103839    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:28.321110 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:28.321027    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4" event={"ID":"94021732-9afe-4fc7-aad6-b71ebbed7af9","Type":"ContainerStarted","Data":"7a0192a97cf28b28f37ff175a650b1e2ef3067d6f5ea53c7683072951092b5a9"}
Sep 11 00:58:28.321324 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:28.321304    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:28.321421 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:28.321378    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:28.323046 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:28.323021    2559 generic.go:356] "Generic (PLEG): container finished" podID="dd3fdc46-edd4-4833-baca-c730c52fed08" containerID="7db57e246bb3c9997005409f6a44e5d8df91e92453511ffefdb43df5beaf12db" exitCode=0
Sep 11 00:58:28.323170 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:28.323108    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tgf62" event={"ID":"dd3fdc46-edd4-4833-baca-c730c52fed08","Type":"ContainerDied","Data":"7db57e246bb3c9997005409f6a44e5d8df91e92453511ffefdb43df5beaf12db"}
Sep 11 00:58:28.337154 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:28.337132    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:28.355339 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:28.355288    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4" podStartSLOduration=9.32252276 podStartE2EDuration="27.35527294s" totalImagesPullingTime="18.03275018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:28.352968711 +0000 UTC m=+27.867568893" watchObservedRunningTime="2026-09-11 00:58:28.35527294 +0000 UTC m=+27.869873122"
Sep 11 00:58:29.107071 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:29.107042    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:29.107824 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:29.107179    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:29.325667 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:29.325636    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:29.342055 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:29.342027    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:58:29.583691 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:29.583493    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7qkdk"]
Sep 11 00:58:29.583833 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:29.583778    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:29.583905 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:29.583887    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:29.586481 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:29.586434    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v55tj"]
Sep 11 00:58:29.586592 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:29.586562    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:29.586706 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:29.586684    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:29.587031 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:29.587008    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gg5rc"]
Sep 11 00:58:29.587142 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:29.587101    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:29.587205 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:29.587187    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:30.329779 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:30.329747    2559 generic.go:356] "Generic (PLEG): container finished" podID="dd3fdc46-edd4-4833-baca-c730c52fed08" containerID="f6daaf7a6801ffc65030617ab1e88fbb1daac3bb55cd7298304e73d6e8ca1aa3" exitCode=0
Sep 11 00:58:30.330261 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:30.329811    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tgf62" event={"ID":"dd3fdc46-edd4-4833-baca-c730c52fed08","Type":"ContainerDied","Data":"f6daaf7a6801ffc65030617ab1e88fbb1daac3bb55cd7298304e73d6e8ca1aa3"}
Sep 11 00:58:31.104634 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:31.104590    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:31.107809 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:31.105002    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:31.107809 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:31.105059    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:31.107809 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:31.105105    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:31.107809 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:31.105193    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:31.107809 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:31.105448    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:32.336663 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:32.336560    2559 generic.go:356] "Generic (PLEG): container finished" podID="dd3fdc46-edd4-4833-baca-c730c52fed08" containerID="da972b2584990f0b0d1b28f3bc44b8d034b2b26785e485e812c40c497b07be26" exitCode=0
Sep 11 00:58:32.336663 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:32.336599    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tgf62" event={"ID":"dd3fdc46-edd4-4833-baca-c730c52fed08","Type":"ContainerDied","Data":"da972b2584990f0b0d1b28f3bc44b8d034b2b26785e485e812c40c497b07be26"}
Sep 11 00:58:33.104643 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:33.104477    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:33.104643 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:33.104498    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:33.104643 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:33.104477    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:33.104643 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:33.104634    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:33.104957 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:33.104692    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:33.104957 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:33.104770    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:34.650083 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:34.650037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:34.650473 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:34.650102    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:34.650473 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:34.650186    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:34.650473 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:34.650205    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:34.650473 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:34.650249    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret podName:2484980c-48af-426f-b3c2-95afc18ed2a2 nodeName:}" failed. No retries permitted until 2026-09-11 00:59:06.650234139 +0000 UTC m=+66.164834299 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret") pod "global-pull-secret-syncer-gg5rc" (UID: "2484980c-48af-426f-b3c2-95afc18ed2a2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 00:58:34.650473 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:34.650264    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs podName:4d739055-efd4-41fe-9725-197913a5257c nodeName:}" failed. No retries permitted until 2026-09-11 00:59:06.650257277 +0000 UTC m=+66.164857437 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs") pod "network-metrics-daemon-7qkdk" (UID: "4d739055-efd4-41fe-9725-197913a5257c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 00:58:34.750965 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:34.750929    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlw2b\" (UniqueName: \"kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b\") pod \"network-check-target-v55tj\" (UID: \"39e30de3-b555-45b2-84b5-0c93e875bddb\") " pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:34.751237 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:34.751125    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 00:58:34.751237 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:34.751152    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 00:58:34.751237 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:34.751167    2559 projected.go:196] Error preparing data for projected volume kube-api-access-dlw2b for pod openshift-network-diagnostics/network-check-target-v55tj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:34.751237 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:34.751236    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b podName:39e30de3-b555-45b2-84b5-0c93e875bddb nodeName:}" failed. No retries permitted until 2026-09-11 00:59:06.751216969 +0000 UTC m=+66.265817147 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-dlw2b" (UniqueName: "kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b") pod "network-check-target-v55tj" (UID: "39e30de3-b555-45b2-84b5-0c93e875bddb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 00:58:35.104670 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.104442    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:35.104859 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.104442    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:35.104859 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:35.104762    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v55tj" podUID="39e30de3-b555-45b2-84b5-0c93e875bddb"
Sep 11 00:58:35.104859 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:35.104850    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gg5rc" podUID="2484980c-48af-426f-b3c2-95afc18ed2a2"
Sep 11 00:58:35.105034 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.104493    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:35.105034 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:35.104965    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7qkdk" podUID="4d739055-efd4-41fe-9725-197913a5257c"
Sep 11 00:58:35.319516 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.319435    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-14.ec2.internal" event="NodeReady"
Sep 11 00:58:35.319688 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.319589    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 00:58:35.352721 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.352681    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74747fb8c-4qgkk"]
Sep 11 00:58:35.355417 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.355388    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.358125 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.358093    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hstjm\""
Sep 11 00:58:35.358125 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.358113    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 00:58:35.358392 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.358376    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 00:58:35.358489 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.358451    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 00:58:35.364322 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.364295    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 00:58:35.365879 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.365858    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qkfrr"]
Sep 11 00:58:35.367898 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.367881    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.369144 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.368997    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tzp2p"]
Sep 11 00:58:35.370402 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.370364    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 00:58:35.370497 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.370434    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 00:58:35.370676 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.370654    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xx7m2\""
Sep 11 00:58:35.371148 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.371128    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74747fb8c-4qgkk"]
Sep 11 00:58:35.371296 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.371219    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tzp2p"
Sep 11 00:58:35.373906 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.373876    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vpvqt\""
Sep 11 00:58:35.374047 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.374028    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 00:58:35.374123 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.374069    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 00:58:35.374123 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.374099    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 00:58:35.382870 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.382825    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qkfrr"]
Sep 11 00:58:35.384222 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.384196    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tzp2p"]
Sep 11 00:58:35.456292 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456243    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh49b\" (UniqueName: \"kubernetes.io/projected/4113c6a8-603c-4e15-a001-047284c67098-kube-api-access-mh49b\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.456292 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456293    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-trusted-ca\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.456545 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456321    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgw75\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-kube-api-access-rgw75\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.456545 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456351    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-certificates\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.456545 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456495    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4113c6a8-603c-4e15-a001-047284c67098-config-volume\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.456545 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.456755 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456561    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f715154-de02-4259-b992-a7acf4e82e5f-cert\") pod \"ingress-canary-tzp2p\" (UID: \"6f715154-de02-4259-b992-a7acf4e82e5f\") " pod="openshift-ingress-canary/ingress-canary-tzp2p"
Sep 11 00:58:35.456755 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456580    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zldmd\" (UniqueName: \"kubernetes.io/projected/6f715154-de02-4259-b992-a7acf4e82e5f-kube-api-access-zldmd\") pod \"ingress-canary-tzp2p\" (UID: \"6f715154-de02-4259-b992-a7acf4e82e5f\") " pod="openshift-ingress-canary/ingress-canary-tzp2p"
Sep 11 00:58:35.456755 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-image-registry-private-configuration\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.456755 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-bound-sa-token\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.456755 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456668    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.456755 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456697    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4113c6a8-603c-4e15-a001-047284c67098-tmp-dir\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.456991 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456751    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c36350a-0286-48a8-9c69-a57d23b2f9bb-ca-trust-extracted\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.456991 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.456787    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-installation-pull-secrets\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.557550 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.557511    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4113c6a8-603c-4e15-a001-047284c67098-config-volume\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.557767 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.557570    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.557767 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.557597    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f715154-de02-4259-b992-a7acf4e82e5f-cert\") pod \"ingress-canary-tzp2p\" (UID: \"6f715154-de02-4259-b992-a7acf4e82e5f\") " pod="openshift-ingress-canary/ingress-canary-tzp2p"
Sep 11 00:58:35.557767 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.557635    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zldmd\" (UniqueName: \"kubernetes.io/projected/6f715154-de02-4259-b992-a7acf4e82e5f-kube-api-access-zldmd\") pod \"ingress-canary-tzp2p\" (UID: \"6f715154-de02-4259-b992-a7acf4e82e5f\") " pod="openshift-ingress-canary/ingress-canary-tzp2p"
Sep 11 00:58:35.557767 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.557658    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-image-registry-private-configuration\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.557767 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.557685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-bound-sa-token\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.557767 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.557711    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.557767 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.557741    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4113c6a8-603c-4e15-a001-047284c67098-tmp-dir\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.558025 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:35.557775    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:35.558025 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:35.557846    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls podName:4113c6a8-603c-4e15-a001-047284c67098 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:36.057826535 +0000 UTC m=+35.572426700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls") pod "dns-default-qkfrr" (UID: "4113c6a8-603c-4e15-a001-047284c67098") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:35.558025 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.557778    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c36350a-0286-48a8-9c69-a57d23b2f9bb-ca-trust-extracted\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.558025 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.557984    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-installation-pull-secrets\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.558152 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.558028    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh49b\" (UniqueName: \"kubernetes.io/projected/4113c6a8-603c-4e15-a001-047284c67098-kube-api-access-mh49b\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.558152 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.558051    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-trusted-ca\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.558152 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.558077    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgw75\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-kube-api-access-rgw75\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.558152 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.558110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-certificates\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.558279 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.558259    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c36350a-0286-48a8-9c69-a57d23b2f9bb-ca-trust-extracted\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.558376 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:35.558361    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 00:58:35.558409 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:35.558381    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74747fb8c-4qgkk: secret "image-registry-tls" not found
Sep 11 00:58:35.558457 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:35.558428    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls podName:7c36350a-0286-48a8-9c69-a57d23b2f9bb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:36.058413295 +0000 UTC m=+35.573013471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls") pod "image-registry-74747fb8c-4qgkk" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb") : secret "image-registry-tls" not found
Sep 11 00:58:35.558457 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.558445    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4113c6a8-603c-4e15-a001-047284c67098-config-volume\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.558719 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.558695    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4113c6a8-603c-4e15-a001-047284c67098-tmp-dir\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.558830 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.558720    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-certificates\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.559294 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.559254    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-trusted-ca\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.562666 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.562640    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-installation-pull-secrets\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.562666 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.562656    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-image-registry-private-configuration\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.562925 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.562727    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f715154-de02-4259-b992-a7acf4e82e5f-cert\") pod \"ingress-canary-tzp2p\" (UID: \"6f715154-de02-4259-b992-a7acf4e82e5f\") " pod="openshift-ingress-canary/ingress-canary-tzp2p"
Sep 11 00:58:35.567374 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.567310    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh49b\" (UniqueName: \"kubernetes.io/projected/4113c6a8-603c-4e15-a001-047284c67098-kube-api-access-mh49b\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:35.567620 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.567579    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgw75\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-kube-api-access-rgw75\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.567721 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.567653    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-bound-sa-token\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:35.567721 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.567707    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zldmd\" (UniqueName: \"kubernetes.io/projected/6f715154-de02-4259-b992-a7acf4e82e5f-kube-api-access-zldmd\") pod \"ingress-canary-tzp2p\" (UID: \"6f715154-de02-4259-b992-a7acf4e82e5f\") " pod="openshift-ingress-canary/ingress-canary-tzp2p"
Sep 11 00:58:35.689293 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:35.689210    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tzp2p"
Sep 11 00:58:36.063019 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:36.062932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:36.063019 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:36.062982    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:36.063248 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:36.063094    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 00:58:36.063248 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:36.063101    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:36.063248 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:36.063108    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74747fb8c-4qgkk: secret "image-registry-tls" not found
Sep 11 00:58:36.063248 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:36.063179    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls podName:4113c6a8-603c-4e15-a001-047284c67098 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:37.063160527 +0000 UTC m=+36.577760708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls") pod "dns-default-qkfrr" (UID: "4113c6a8-603c-4e15-a001-047284c67098") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:36.063248 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:36.063234    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls podName:7c36350a-0286-48a8-9c69-a57d23b2f9bb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:37.063211089 +0000 UTC m=+36.577811272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls") pod "image-registry-74747fb8c-4qgkk" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb") : secret "image-registry-tls" not found
Sep 11 00:58:36.640688 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:36.640619    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tzp2p"]
Sep 11 00:58:36.652496 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:58:36.652467    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f715154_de02_4259_b992_a7acf4e82e5f.slice/crio-0bfb5a5020cbdbc4fd720f05e3c87d1deda73a8847d00acd6b3f1c6250586664 WatchSource:0}: Error finding container 0bfb5a5020cbdbc4fd720f05e3c87d1deda73a8847d00acd6b3f1c6250586664: Status 404 returned error can't find the container with id 0bfb5a5020cbdbc4fd720f05e3c87d1deda73a8847d00acd6b3f1c6250586664
Sep 11 00:58:37.069474 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.069374    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:37.069474 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.069479    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:37.070185 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:37.069538    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 00:58:37.070185 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:37.069561    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:37.070185 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:37.069623    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls podName:4113c6a8-603c-4e15-a001-047284c67098 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:39.069597301 +0000 UTC m=+38.584197460 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls") pod "dns-default-qkfrr" (UID: "4113c6a8-603c-4e15-a001-047284c67098") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:37.070185 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:37.069561    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74747fb8c-4qgkk: secret "image-registry-tls" not found
Sep 11 00:58:37.070185 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:37.069680    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls podName:7c36350a-0286-48a8-9c69-a57d23b2f9bb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:39.069666871 +0000 UTC m=+38.584267060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls") pod "image-registry-74747fb8c-4qgkk" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb") : secret "image-registry-tls" not found
Sep 11 00:58:37.103991 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.103955    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:58:37.103991 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.103995    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:58:37.104249 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.104190    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:58:37.106767 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.106745    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 00:58:37.107862 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.107796    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 00:58:37.107862 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.107822    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 00:58:37.108123 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.108105    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h86gq\""
Sep 11 00:58:37.108212 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.108125    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zpsrm\""
Sep 11 00:58:37.108212 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.108143    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 00:58:37.352492 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.352414    2559 generic.go:356] "Generic (PLEG): container finished" podID="dd3fdc46-edd4-4833-baca-c730c52fed08" containerID="0884bac6b13abf8f0ae0fbd95a2e2a29f638b9e3d4688d89535d31b1f6d2c59b" exitCode=0
Sep 11 00:58:37.352661 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.352502    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tgf62" event={"ID":"dd3fdc46-edd4-4833-baca-c730c52fed08","Type":"ContainerDied","Data":"0884bac6b13abf8f0ae0fbd95a2e2a29f638b9e3d4688d89535d31b1f6d2c59b"}
Sep 11 00:58:37.353789 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:37.353767    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tzp2p" event={"ID":"6f715154-de02-4259-b992-a7acf4e82e5f","Type":"ContainerStarted","Data":"0bfb5a5020cbdbc4fd720f05e3c87d1deda73a8847d00acd6b3f1c6250586664"}
Sep 11 00:58:38.360516 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:38.360292    2559 generic.go:356] "Generic (PLEG): container finished" podID="dd3fdc46-edd4-4833-baca-c730c52fed08" containerID="536e891af20826e2aa5d5f83a4ae4eb59d94a7fc8ccee7bb7fb31d7100c2d1b3" exitCode=0
Sep 11 00:58:38.360516 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:38.360372    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tgf62" event={"ID":"dd3fdc46-edd4-4833-baca-c730c52fed08","Type":"ContainerDied","Data":"536e891af20826e2aa5d5f83a4ae4eb59d94a7fc8ccee7bb7fb31d7100c2d1b3"}
Sep 11 00:58:39.086033 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:39.085944    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:39.086033 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:39.085987    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:39.086236 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:39.086100    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:39.086236 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:39.086141    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 00:58:39.086236 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:39.086152    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74747fb8c-4qgkk: secret "image-registry-tls" not found
Sep 11 00:58:39.086236 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:39.086165    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls podName:4113c6a8-603c-4e15-a001-047284c67098 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:43.086149854 +0000 UTC m=+42.600750014 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls") pod "dns-default-qkfrr" (UID: "4113c6a8-603c-4e15-a001-047284c67098") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:39.086236 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:39.086189    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls podName:7c36350a-0286-48a8-9c69-a57d23b2f9bb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:43.086178631 +0000 UTC m=+42.600778791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls") pod "image-registry-74747fb8c-4qgkk" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb") : secret "image-registry-tls" not found
Sep 11 00:58:39.366761 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:39.366725    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tgf62" event={"ID":"dd3fdc46-edd4-4833-baca-c730c52fed08","Type":"ContainerStarted","Data":"e287c6bc752de7505a0c74f030de637c12858b600c01a972ec792495bb1a22a8"}
Sep 11 00:58:39.368074 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:39.368049    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tzp2p" event={"ID":"6f715154-de02-4259-b992-a7acf4e82e5f","Type":"ContainerStarted","Data":"e2ca9c8b66e710afa19bf41396d6609e3f51da96c2a0d6be5ad2b3c684d9d83c"}
Sep 11 00:58:39.391061 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:39.391014    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tgf62" podStartSLOduration=9.411075344 podStartE2EDuration="38.390999689s" totalImagesPullingTime="28.979924345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:39.390331942 +0000 UTC m=+38.904932128" watchObservedRunningTime="2026-09-11 00:58:39.390999689 +0000 UTC m=+38.905599881"
Sep 11 00:58:39.406135 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:39.406088    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tzp2p" podStartSLOduration=2.27632267 podStartE2EDuration="4.406074084s" totalImagesPullingTime="2.129751414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:58:39.40556424 +0000 UTC m=+38.920164421" watchObservedRunningTime="2026-09-11 00:58:39.406074084 +0000 UTC m=+38.920674266"
Sep 11 00:58:42.777005 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.776967    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jfgl6"]
Sep 11 00:58:42.809010 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.808973    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jfgl6"]
Sep 11 00:58:42.809155 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.809090    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:42.811597 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.811571    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 00:58:42.811744 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.811660    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 00:58:42.811744 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.811692    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 00:58:42.812787 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.812769    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 00:58:42.812877 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.812803    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mtm2j\""
Sep 11 00:58:42.915200 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.915158    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:42.915372 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.915223    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt62v\" (UniqueName: \"kubernetes.io/projected/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-api-access-qt62v\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:42.915372 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.915312    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-crio-socket\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:42.915372 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.915346    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:42.915475 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:42.915415    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-data-volume\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.016081 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.016047    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt62v\" (UniqueName: \"kubernetes.io/projected/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-api-access-qt62v\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.016223 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.016104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-crio-socket\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.016223 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.016132    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.016223 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.016172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-data-volume\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.016333 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.016225    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.016333 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.016277    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:43.016333 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.016328    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 00:58:43.016436 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.016333    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-crio-socket\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.016436 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.016363    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:43.516341419 +0000 UTC m=+43.030941598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:43.016436 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.016391    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:43.516376244 +0000 UTC m=+43.030976422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : configmap "kube-rbac-proxy" not found
Sep 11 00:58:43.016638 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.016592    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-data-volume\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.026335 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.026314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt62v\" (UniqueName: \"kubernetes.io/projected/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-api-access-qt62v\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.116781 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.116697    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:43.116925 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.116851    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 00:58:43.116925 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.116867    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74747fb8c-4qgkk: secret "image-registry-tls" not found
Sep 11 00:58:43.116925 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.116874    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:43.116925 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.116922    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls podName:7c36350a-0286-48a8-9c69-a57d23b2f9bb nodeName:}" failed. No retries permitted until 2026-09-11 00:58:51.116907554 +0000 UTC m=+50.631507735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls") pod "image-registry-74747fb8c-4qgkk" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb") : secret "image-registry-tls" not found
Sep 11 00:58:43.117072 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.116974    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:43.117072 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.117044    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls podName:4113c6a8-603c-4e15-a001-047284c67098 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:51.117007148 +0000 UTC m=+50.631607307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls") pod "dns-default-qkfrr" (UID: "4113c6a8-603c-4e15-a001-047284c67098") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:43.520695 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.520664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.520843 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:43.520728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:43.520843 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.520792    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 00:58:43.520843 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.520817    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:43.520843 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.520841    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:44.520828958 +0000 UTC m=+44.035429132 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : configmap "kube-rbac-proxy" not found
Sep 11 00:58:43.520984 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:43.520867    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:44.520854823 +0000 UTC m=+44.035454988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:44.528459 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:44.528420    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:44.528921 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:44.528507    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:44.528921 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:44.528548    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 00:58:44.528921 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:44.528631    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:46.528616524 +0000 UTC m=+46.043216697 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : configmap "kube-rbac-proxy" not found
Sep 11 00:58:44.528921 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:44.528704    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:44.528921 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:44.528782    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:46.528760579 +0000 UTC m=+46.043360757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:44.731202 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:44.731170    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-twtj4_a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec/dns-node-resolver/0.log"
Sep 11 00:58:45.930412 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:45.930385    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hrx2c_0a670085-8845-40b7-ab78-2d028806840c/node-ca/0.log"
Sep 11 00:58:46.545390 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:46.545354    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:46.545581 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:46.545426    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:46.545581 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:46.545513    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 00:58:46.545581 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:46.545523    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:46.545581 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:46.545568    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:50.545555085 +0000 UTC m=+50.060155246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : configmap "kube-rbac-proxy" not found
Sep 11 00:58:46.545581 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:46.545581    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:50.545574694 +0000 UTC m=+50.060174854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:46.730788 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:46.730758    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tzp2p_6f715154-de02-4259-b992-a7acf4e82e5f/serve-healthcheck-canary/0.log"
Sep 11 00:58:50.575859 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:50.575822    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:50.576238 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:50.575910    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:50.576238 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:50.575934    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 00:58:50.576238 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:50.575989    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:58.575973664 +0000 UTC m=+58.090573840 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : configmap "kube-rbac-proxy" not found
Sep 11 00:58:50.576238 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:50.576025    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:50.576238 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:50.576086    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:58:58.576070664 +0000 UTC m=+58.090670844 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:51.178622 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:51.178577    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:58:51.178786 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:51.178634    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls\") pod \"image-registry-74747fb8c-4qgkk\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") " pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:58:51.178786 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:51.178715    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 00:58:51.178786 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:51.178726    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74747fb8c-4qgkk: secret "image-registry-tls" not found
Sep 11 00:58:51.178786 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:51.178767    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls podName:7c36350a-0286-48a8-9c69-a57d23b2f9bb nodeName:}" failed. No retries permitted until 2026-09-11 00:59:07.178754837 +0000 UTC m=+66.693355012 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls") pod "image-registry-74747fb8c-4qgkk" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb") : secret "image-registry-tls" not found
Sep 11 00:58:51.178923 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:51.178804    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 00:58:51.178923 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:51.178851    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls podName:4113c6a8-603c-4e15-a001-047284c67098 nodeName:}" failed. No retries permitted until 2026-09-11 00:59:07.178839079 +0000 UTC m=+66.693439239 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls") pod "dns-default-qkfrr" (UID: "4113c6a8-603c-4e15-a001-047284c67098") : secret "dns-default-metrics-tls" not found
Sep 11 00:58:58.631125 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:58.631079    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:58.631569 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:58.631184    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:58:58.631569 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:58.631337    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:58.631569 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:58:58.631404    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls podName:efecf1f1-ec51-4c26-bd4a-56f81bce1f52 nodeName:}" failed. No retries permitted until 2026-09-11 00:59:14.631388933 +0000 UTC m=+74.145989103 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jfgl6" (UID: "efecf1f1-ec51-4c26-bd4a-56f81bce1f52") : secret "insights-runtime-extractor-tls" not found
Sep 11 00:58:58.631720 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:58:58.631685    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:59:01.348420 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:01.348382    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lr2p4"
Sep 11 00:59:05.123186 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.123151    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74747fb8c-4qgkk"]
Sep 11 00:59:05.123559 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:59:05.123343    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-74747fb8c-4qgkk" podUID="7c36350a-0286-48a8-9c69-a57d23b2f9bb"
Sep 11 00:59:05.127575 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.127535    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz"]
Sep 11 00:59:05.130456 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.130436    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-p5cwr"]
Sep 11 00:59:05.130573 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.130559    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz"
Sep 11 00:59:05.132068 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.132027    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-p5cwr"
Sep 11 00:59:05.133450 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.133433    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 00:59:05.133551 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.133434    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-c7bfn\""
Sep 11 00:59:05.134168 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.134147    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 00:59:05.138148 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.138128    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-s2xx8\""
Sep 11 00:59:05.138222 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.138134    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 00:59:05.144445 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.144425    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz"]
Sep 11 00:59:05.147668 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.147583    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-p5cwr"]
Sep 11 00:59:05.185043 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.185011    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmdp8\" (UniqueName: \"kubernetes.io/projected/b8c6b379-1a03-4d54-ad83-ae2217a4df4f-kube-api-access-lmdp8\") pod \"downloads-76f99584f4-p5cwr\" (UID: \"b8c6b379-1a03-4d54-ad83-ae2217a4df4f\") " pod="openshift-console/downloads-76f99584f4-p5cwr"
Sep 11 00:59:05.185281 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.185178    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/111574e8-558e-4a8d-bd09-cd35c6a2aba1-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-zp4wz\" (UID: \"111574e8-558e-4a8d-bd09-cd35c6a2aba1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz"
Sep 11 00:59:05.286096 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.286050    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/111574e8-558e-4a8d-bd09-cd35c6a2aba1-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-zp4wz\" (UID: \"111574e8-558e-4a8d-bd09-cd35c6a2aba1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz"
Sep 11 00:59:05.286283 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.286120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmdp8\" (UniqueName: \"kubernetes.io/projected/b8c6b379-1a03-4d54-ad83-ae2217a4df4f-kube-api-access-lmdp8\") pod \"downloads-76f99584f4-p5cwr\" (UID: \"b8c6b379-1a03-4d54-ad83-ae2217a4df4f\") " pod="openshift-console/downloads-76f99584f4-p5cwr"
Sep 11 00:59:05.288415 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.288391    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/111574e8-558e-4a8d-bd09-cd35c6a2aba1-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-zp4wz\" (UID: \"111574e8-558e-4a8d-bd09-cd35c6a2aba1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz"
Sep 11 00:59:05.295927 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.295901    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmdp8\" (UniqueName: \"kubernetes.io/projected/b8c6b379-1a03-4d54-ad83-ae2217a4df4f-kube-api-access-lmdp8\") pod \"downloads-76f99584f4-p5cwr\" (UID: \"b8c6b379-1a03-4d54-ad83-ae2217a4df4f\") " pod="openshift-console/downloads-76f99584f4-p5cwr"
Sep 11 00:59:05.430638 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.430545    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:59:05.436192 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.436171    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:59:05.440032 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.440017    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz"
Sep 11 00:59:05.444587 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.444565    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-p5cwr"
Sep 11 00:59:05.487891 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.487811    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-bound-sa-token\") pod \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") "
Sep 11 00:59:05.487891 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.487848    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c36350a-0286-48a8-9c69-a57d23b2f9bb-ca-trust-extracted\") pod \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") "
Sep 11 00:59:05.487891 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.487882    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-trusted-ca\") pod \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") "
Sep 11 00:59:05.488113 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.487918    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-installation-pull-secrets\") pod \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") "
Sep 11 00:59:05.488113 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.487948    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-certificates\") pod \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") "
Sep 11 00:59:05.488113 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.487989    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-image-registry-private-configuration\") pod \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") "
Sep 11 00:59:05.488113 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.488018    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgw75\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-kube-api-access-rgw75\") pod \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\" (UID: \"7c36350a-0286-48a8-9c69-a57d23b2f9bb\") "
Sep 11 00:59:05.488634 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.488588    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7c36350a-0286-48a8-9c69-a57d23b2f9bb" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:59:05.489009 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.488959    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c36350a-0286-48a8-9c69-a57d23b2f9bb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7c36350a-0286-48a8-9c69-a57d23b2f9bb" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:59:05.489224 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.489188    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7c36350a-0286-48a8-9c69-a57d23b2f9bb" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:59:05.491067 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.491025    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7c36350a-0286-48a8-9c69-a57d23b2f9bb" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:59:05.491222 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.491175    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7c36350a-0286-48a8-9c69-a57d23b2f9bb" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:59:05.493228 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.493046    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7c36350a-0286-48a8-9c69-a57d23b2f9bb" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:59:05.493906 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.493879    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-kube-api-access-rgw75" (OuterVolumeSpecName: "kube-api-access-rgw75") pod "7c36350a-0286-48a8-9c69-a57d23b2f9bb" (UID: "7c36350a-0286-48a8-9c69-a57d23b2f9bb"). InnerVolumeSpecName "kube-api-access-rgw75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:59:05.588762 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.588725    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-bound-sa-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:05.588762 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.588753    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7c36350a-0286-48a8-9c69-a57d23b2f9bb-ca-trust-extracted\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:05.588762 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.588763    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-trusted-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:05.588762 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.588773    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-installation-pull-secrets\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:05.589020 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.588782    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-certificates\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:05.589020 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.588811    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7c36350a-0286-48a8-9c69-a57d23b2f9bb-image-registry-private-configuration\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:05.589020 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.588820    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgw75\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-kube-api-access-rgw75\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:05.712741 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.712665    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz"]
Sep 11 00:59:05.716812 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:05.716138    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111574e8_558e_4a8d_bd09_cd35c6a2aba1.slice/crio-e2e92c15a0bb30e7fb91973a298ea87403eccfaa0ee3db24ced2a8c6deb9c332 WatchSource:0}: Error finding container e2e92c15a0bb30e7fb91973a298ea87403eccfaa0ee3db24ced2a8c6deb9c332: Status 404 returned error can't find the container with id e2e92c15a0bb30e7fb91973a298ea87403eccfaa0ee3db24ced2a8c6deb9c332
Sep 11 00:59:05.718151 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:05.717419    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-p5cwr"]
Sep 11 00:59:05.720394 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:05.720373    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c6b379_1a03_4d54_ad83_ae2217a4df4f.slice/crio-2d77f87c98e5247da522fc2a3da02feee2531a7a37c09de130edd77266318b79 WatchSource:0}: Error finding container 2d77f87c98e5247da522fc2a3da02feee2531a7a37c09de130edd77266318b79: Status 404 returned error can't find the container with id 2d77f87c98e5247da522fc2a3da02feee2531a7a37c09de130edd77266318b79
Sep 11 00:59:06.435032 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.434999    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-p5cwr" event={"ID":"b8c6b379-1a03-4d54-ad83-ae2217a4df4f","Type":"ContainerStarted","Data":"2d77f87c98e5247da522fc2a3da02feee2531a7a37c09de130edd77266318b79"}
Sep 11 00:59:06.436430 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.436403    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74747fb8c-4qgkk"
Sep 11 00:59:06.436548 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.436407    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz" event={"ID":"111574e8-558e-4a8d-bd09-cd35c6a2aba1","Type":"ContainerStarted","Data":"e2e92c15a0bb30e7fb91973a298ea87403eccfaa0ee3db24ced2a8c6deb9c332"}
Sep 11 00:59:06.471816 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.471780    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74747fb8c-4qgkk"]
Sep 11 00:59:06.476102 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.476075    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74747fb8c-4qgkk"]
Sep 11 00:59:06.598516 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.598435    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7c36350a-0286-48a8-9c69-a57d23b2f9bb-registry-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:06.699724 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.699695    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:59:06.699875 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.699751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:59:06.706683 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.706652    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 00:59:06.706683 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.706652    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 00:59:06.712231 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.712211    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2484980c-48af-426f-b3c2-95afc18ed2a2-original-pull-secret\") pod \"global-pull-secret-syncer-gg5rc\" (UID: \"2484980c-48af-426f-b3c2-95afc18ed2a2\") " pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:59:06.712368 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.712348    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d739055-efd4-41fe-9725-197913a5257c-metrics-certs\") pod \"network-metrics-daemon-7qkdk\" (UID: \"4d739055-efd4-41fe-9725-197913a5257c\") " pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:59:06.800930 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.800885    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlw2b\" (UniqueName: \"kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b\") pod \"network-check-target-v55tj\" (UID: \"39e30de3-b555-45b2-84b5-0c93e875bddb\") " pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:59:06.803848 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.803827    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 00:59:06.822248 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.822220    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zpsrm\""
Sep 11 00:59:06.822373 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.822260    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 00:59:06.827398 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.827381    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7qkdk"
Sep 11 00:59:06.829127 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.829103    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gg5rc"
Sep 11 00:59:06.835253 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:06.835219    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlw2b\" (UniqueName: \"kubernetes.io/projected/39e30de3-b555-45b2-84b5-0c93e875bddb-kube-api-access-dlw2b\") pod \"network-check-target-v55tj\" (UID: \"39e30de3-b555-45b2-84b5-0c93e875bddb\") " pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:59:07.102222 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.102181    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gg5rc"]
Sep 11 00:59:07.106482 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:07.106450    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2484980c_48af_426f_b3c2_95afc18ed2a2.slice/crio-802a9f24f2d51b04a7abe15c0a01624b7dc5ed7ad33a394716aaf42970281fbc WatchSource:0}: Error finding container 802a9f24f2d51b04a7abe15c0a01624b7dc5ed7ad33a394716aaf42970281fbc: Status 404 returned error can't find the container with id 802a9f24f2d51b04a7abe15c0a01624b7dc5ed7ad33a394716aaf42970281fbc
Sep 11 00:59:07.109863 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.109834    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c36350a-0286-48a8-9c69-a57d23b2f9bb" path="/var/lib/kubelet/pods/7c36350a-0286-48a8-9c69-a57d23b2f9bb/volumes"
Sep 11 00:59:07.110061 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.110043    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7qkdk"]
Sep 11 00:59:07.110876 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:07.110849    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d739055_efd4_41fe_9725_197913a5257c.slice/crio-16438b89bfa7523ae1ed4eaafd484560e9443fe442fd93a950c0a0b0385f12c4 WatchSource:0}: Error finding container 16438b89bfa7523ae1ed4eaafd484560e9443fe442fd93a950c0a0b0385f12c4: Status 404 returned error can't find the container with id 16438b89bfa7523ae1ed4eaafd484560e9443fe442fd93a950c0a0b0385f12c4
Sep 11 00:59:07.126389 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.126350    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h86gq\""
Sep 11 00:59:07.134121 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.134045    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:59:07.203148 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.203103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:59:07.205828 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.205793    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4113c6a8-603c-4e15-a001-047284c67098-metrics-tls\") pod \"dns-default-qkfrr\" (UID: \"4113c6a8-603c-4e15-a001-047284c67098\") " pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:59:07.415133 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.415053    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v55tj"]
Sep 11 00:59:07.418337 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:07.418293    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e30de3_b555_45b2_84b5_0c93e875bddb.slice/crio-5d57338de4d3fa9484047523bd9dba3cbaa1029ecaa227cd58bebf987ad7c942 WatchSource:0}: Error finding container 5d57338de4d3fa9484047523bd9dba3cbaa1029ecaa227cd58bebf987ad7c942: Status 404 returned error can't find the container with id 5d57338de4d3fa9484047523bd9dba3cbaa1029ecaa227cd58bebf987ad7c942
Sep 11 00:59:07.440417 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.440375    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v55tj" event={"ID":"39e30de3-b555-45b2-84b5-0c93e875bddb","Type":"ContainerStarted","Data":"5d57338de4d3fa9484047523bd9dba3cbaa1029ecaa227cd58bebf987ad7c942"}
Sep 11 00:59:07.442195 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.442170    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz" event={"ID":"111574e8-558e-4a8d-bd09-cd35c6a2aba1","Type":"ContainerStarted","Data":"a4ee082355f9e60b7b73cb6a3d495daaff1985478437df11cabf85a428333fb9"}
Sep 11 00:59:07.442410 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.442386    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz"
Sep 11 00:59:07.443427 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.443399    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gg5rc" event={"ID":"2484980c-48af-426f-b3c2-95afc18ed2a2","Type":"ContainerStarted","Data":"802a9f24f2d51b04a7abe15c0a01624b7dc5ed7ad33a394716aaf42970281fbc"}
Sep 11 00:59:07.444755 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.444732    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7qkdk" event={"ID":"4d739055-efd4-41fe-9725-197913a5257c","Type":"ContainerStarted","Data":"16438b89bfa7523ae1ed4eaafd484560e9443fe442fd93a950c0a0b0385f12c4"}
Sep 11 00:59:07.448223 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.448204    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz"
Sep 11 00:59:07.458254 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.458206    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zp4wz" podStartSLOduration=1.527133514 podStartE2EDuration="2.458187656s" totalImagesPullingTime="931.054142ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:07.457715035 +0000 UTC m=+66.972315223" watchObservedRunningTime="2026-09-11 00:59:07.458187656 +0000 UTC m=+66.972787838"
Sep 11 00:59:07.482725 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.482696    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xx7m2\""
Sep 11 00:59:07.491149 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.491118    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:59:07.794432 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:07.794342    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qkfrr"]
Sep 11 00:59:08.282715 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.282683    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cd56bb784-gtbv6"]
Sep 11 00:59:08.303968 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.303935    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd56bb784-gtbv6"]
Sep 11 00:59:08.304144 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.304059    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.307246 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.307222    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 00:59:08.307389 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.307222    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 00:59:08.307520 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.307440    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-w68nv\""
Sep 11 00:59:08.307662 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.307580    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 00:59:08.307662 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.307657    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 00:59:08.308725 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.308529    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 00:59:08.413115 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.413080    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-console-config\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.413303 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.413129    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-serving-cert\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.413303 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.413158    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-oauth-config\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.413303 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.413194    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-service-ca\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.413303 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.413231    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-oauth-serving-cert\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.413303 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.413286    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nhp6\" (UniqueName: \"kubernetes.io/projected/6b8ce73c-a602-4633-9a36-88482852b2cf-kube-api-access-7nhp6\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.436921 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.436412    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"]
Sep 11 00:59:08.461887 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.461850    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"]
Sep 11 00:59:08.462335 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.461900    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qkfrr" event={"ID":"4113c6a8-603c-4e15-a001-047284c67098","Type":"ContainerStarted","Data":"b5e3b77612cd14657b31178611f6b972f395af416fc78355f5f5e08bd7ad4fd1"}
Sep 11 00:59:08.462335 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.462007    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.465938 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.465914    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 11 00:59:08.466237 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.466221    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 00:59:08.466354 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.466333    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 00:59:08.466439 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.466424    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 11 00:59:08.466567 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.466551    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 00:59:08.466657 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.466643    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jbsqv\""
Sep 11 00:59:08.515157 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.514149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-serving-cert\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.515157 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.514195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-oauth-config\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.515157 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.514233    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dff58d4d-619a-422a-9f40-76ff0940abde-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.515157 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.514282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dff58d4d-619a-422a-9f40-76ff0940abde-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.515157 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.514315    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-service-ca\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.515157 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.514338    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tgbq\" (UniqueName: \"kubernetes.io/projected/dff58d4d-619a-422a-9f40-76ff0940abde-kube-api-access-2tgbq\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.515157 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.514401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-oauth-serving-cert\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.515157 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.514445    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dff58d4d-619a-422a-9f40-76ff0940abde-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.515157 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.514491    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nhp6\" (UniqueName: \"kubernetes.io/projected/6b8ce73c-a602-4633-9a36-88482852b2cf-kube-api-access-7nhp6\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.515157 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.514530    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-console-config\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.515817 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.515274    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-service-ca\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.515817 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.515281    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-console-config\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.515949 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.515927    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-oauth-serving-cert\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.518622 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.518559    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-serving-cert\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.518764 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.518687    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-oauth-config\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.525289 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.525262    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nhp6\" (UniqueName: \"kubernetes.io/projected/6b8ce73c-a602-4633-9a36-88482852b2cf-kube-api-access-7nhp6\") pod \"console-cd56bb784-gtbv6\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") " pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.615667 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.615564    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dff58d4d-619a-422a-9f40-76ff0940abde-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.615667 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.615639    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dff58d4d-619a-422a-9f40-76ff0940abde-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.615874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.615671    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tgbq\" (UniqueName: \"kubernetes.io/projected/dff58d4d-619a-422a-9f40-76ff0940abde-kube-api-access-2tgbq\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.615874 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.615719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dff58d4d-619a-422a-9f40-76ff0940abde-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.615874 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:59:08.615851    2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 11 00:59:08.616023 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:59:08.615916    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dff58d4d-619a-422a-9f40-76ff0940abde-prometheus-operator-tls podName:dff58d4d-619a-422a-9f40-76ff0940abde nodeName:}" failed. No retries permitted until 2026-09-11 00:59:09.115896281 +0000 UTC m=+68.630496464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/dff58d4d-619a-422a-9f40-76ff0940abde-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-bn4l7" (UID: "dff58d4d-619a-422a-9f40-76ff0940abde") : secret "prometheus-operator-tls" not found
Sep 11 00:59:08.616598 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.616560    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dff58d4d-619a-422a-9f40-76ff0940abde-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.618684 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.618424    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:08.619098 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.619052    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dff58d4d-619a-422a-9f40-76ff0940abde-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:08.625436 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:08.625404    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tgbq\" (UniqueName: \"kubernetes.io/projected/dff58d4d-619a-422a-9f40-76ff0940abde-kube-api-access-2tgbq\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:09.085051 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:09.084287    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd56bb784-gtbv6"]
Sep 11 00:59:09.090274 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:09.090238    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8ce73c_a602_4633_9a36_88482852b2cf.slice/crio-89231431d2b01a7b0f829d8a7719cb15de088aff42167abfe5520c2ad83b0734 WatchSource:0}: Error finding container 89231431d2b01a7b0f829d8a7719cb15de088aff42167abfe5520c2ad83b0734: Status 404 returned error can't find the container with id 89231431d2b01a7b0f829d8a7719cb15de088aff42167abfe5520c2ad83b0734
Sep 11 00:59:09.119289 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:09.119252    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dff58d4d-619a-422a-9f40-76ff0940abde-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:09.122671 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:09.122643    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dff58d4d-619a-422a-9f40-76ff0940abde-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-bn4l7\" (UID: \"dff58d4d-619a-422a-9f40-76ff0940abde\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:09.376591 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:09.376555    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"
Sep 11 00:59:09.464041 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:09.463954    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd56bb784-gtbv6" event={"ID":"6b8ce73c-a602-4633-9a36-88482852b2cf","Type":"ContainerStarted","Data":"89231431d2b01a7b0f829d8a7719cb15de088aff42167abfe5520c2ad83b0734"}
Sep 11 00:59:09.470098 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:09.469687    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7qkdk" event={"ID":"4d739055-efd4-41fe-9725-197913a5257c","Type":"ContainerStarted","Data":"2794c7c2bac991567364ec6777b27779abc541fa69fa5531d9b08e1b997393df"}
Sep 11 00:59:09.470098 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:09.469750    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7qkdk" event={"ID":"4d739055-efd4-41fe-9725-197913a5257c","Type":"ContainerStarted","Data":"4a22471ee81a50f4622c12d0c0e2bf54c39aa449553079fc5d351a3a4e4fb6c4"}
Sep 11 00:59:09.490814 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:09.489396    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7qkdk" podStartSLOduration=66.801556118 podStartE2EDuration="1m8.489373449s" totalImagesPullingTime="1.687817331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:09.48836911 +0000 UTC m=+69.002969285" watchObservedRunningTime="2026-09-11 00:59:09.489373449 +0000 UTC m=+69.003973633"
Sep 11 00:59:09.695119 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:09.695086    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7"]
Sep 11 00:59:09.700132 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:09.700099    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddff58d4d_619a_422a_9f40_76ff0940abde.slice/crio-873ec3ffc20c4542f3190d9d2457149bf11695f7b2b6bdb695563358bede8cd8 WatchSource:0}: Error finding container 873ec3ffc20c4542f3190d9d2457149bf11695f7b2b6bdb695563358bede8cd8: Status 404 returned error can't find the container with id 873ec3ffc20c4542f3190d9d2457149bf11695f7b2b6bdb695563358bede8cd8
Sep 11 00:59:10.477800 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.477714    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7" event={"ID":"dff58d4d-619a-422a-9f40-76ff0940abde","Type":"ContainerStarted","Data":"873ec3ffc20c4542f3190d9d2457149bf11695f7b2b6bdb695563358bede8cd8"}
Sep 11 00:59:10.672998 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.672956    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-784ffbcd54-ccvhc"]
Sep 11 00:59:10.675559 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.675526    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.690699 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.690454    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 00:59:10.691436 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.691414    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784ffbcd54-ccvhc"]
Sep 11 00:59:10.735370 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.735046    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-oauth-serving-cert\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.735370 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.735107    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfxm8\" (UniqueName: \"kubernetes.io/projected/0481f0ef-71c9-4dae-86ff-13510383e309-kube-api-access-zfxm8\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.735370 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.735145    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-console-config\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.735370 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.735171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-serving-cert\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.735370 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.735194    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-oauth-config\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.735370 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.735229    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-service-ca\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.735370 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.735256    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-trusted-ca-bundle\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.835893 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.835809    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-oauth-serving-cert\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.835893 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.835874    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfxm8\" (UniqueName: \"kubernetes.io/projected/0481f0ef-71c9-4dae-86ff-13510383e309-kube-api-access-zfxm8\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.836138 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.835909    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-console-config\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.836138 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.835934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-serving-cert\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.836138 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.835959    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-oauth-config\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.836138 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.835992    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-service-ca\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.836138 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.836025    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-trusted-ca-bundle\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.837011 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.836693    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-oauth-serving-cert\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.837187 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.837164    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-trusted-ca-bundle\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.837701 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.837670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-service-ca\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.837808 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.837685    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-console-config\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.839195 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.839155    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-oauth-config\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.840840 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.840803    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-serving-cert\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.845530 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.845503    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfxm8\" (UniqueName: \"kubernetes.io/projected/0481f0ef-71c9-4dae-86ff-13510383e309-kube-api-access-zfxm8\") pod \"console-784ffbcd54-ccvhc\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") " pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:10.990924 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:10.990827    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:14.667578 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:14.667533    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:59:14.670197 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:14.670170    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/efecf1f1-ec51-4c26-bd4a-56f81bce1f52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jfgl6\" (UID: \"efecf1f1-ec51-4c26-bd4a-56f81bce1f52\") " pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:59:14.921527 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:14.921446    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mtm2j\""
Sep 11 00:59:14.928701 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:14.928665    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jfgl6"
Sep 11 00:59:15.728268 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:15.728242    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jfgl6"]
Sep 11 00:59:15.731815 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:15.731782    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefecf1f1_ec51_4c26_bd4a_56f81bce1f52.slice/crio-e4beaa86853e69f32f7216b585cf6c71e5e9e428fcee5a5dd473e26c0b07a495 WatchSource:0}: Error finding container e4beaa86853e69f32f7216b585cf6c71e5e9e428fcee5a5dd473e26c0b07a495: Status 404 returned error can't find the container with id e4beaa86853e69f32f7216b585cf6c71e5e9e428fcee5a5dd473e26c0b07a495
Sep 11 00:59:15.740817 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:15.740636    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-784ffbcd54-ccvhc"]
Sep 11 00:59:15.749506 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:15.749455    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0481f0ef_71c9_4dae_86ff_13510383e309.slice/crio-386160f8a5cb463610956deed9e176f049d271e19e32f893cd59e374198b959b WatchSource:0}: Error finding container 386160f8a5cb463610956deed9e176f049d271e19e32f893cd59e374198b959b: Status 404 returned error can't find the container with id 386160f8a5cb463610956deed9e176f049d271e19e32f893cd59e374198b959b
Sep 11 00:59:16.517546 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.517495    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd56bb784-gtbv6" event={"ID":"6b8ce73c-a602-4633-9a36-88482852b2cf","Type":"ContainerStarted","Data":"592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229"}
Sep 11 00:59:16.519735 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.519708    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7" event={"ID":"dff58d4d-619a-422a-9f40-76ff0940abde","Type":"ContainerStarted","Data":"3e7c34acf1fd005efe42a56f3e680372bd2baa41eeb8412bda299c3144e2d874"}
Sep 11 00:59:16.519854 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.519762    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7" event={"ID":"dff58d4d-619a-422a-9f40-76ff0940abde","Type":"ContainerStarted","Data":"8a6cb671fade87486790c04c8f2d37eaa080ca1a0a18bfcb1ffd2abf5037ec9b"}
Sep 11 00:59:16.521409 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.521376    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v55tj" event={"ID":"39e30de3-b555-45b2-84b5-0c93e875bddb","Type":"ContainerStarted","Data":"90c6d2540ab99f1589500acfb74aed4eb9577810936112478952d32b7336d51a"}
Sep 11 00:59:16.521513 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.521478    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:59:16.523132 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.523109    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784ffbcd54-ccvhc" event={"ID":"0481f0ef-71c9-4dae-86ff-13510383e309","Type":"ContainerStarted","Data":"e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8"}
Sep 11 00:59:16.523243 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.523155    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784ffbcd54-ccvhc" event={"ID":"0481f0ef-71c9-4dae-86ff-13510383e309","Type":"ContainerStarted","Data":"386160f8a5cb463610956deed9e176f049d271e19e32f893cd59e374198b959b"}
Sep 11 00:59:16.525198 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.525176    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jfgl6" event={"ID":"efecf1f1-ec51-4c26-bd4a-56f81bce1f52","Type":"ContainerStarted","Data":"9480e63acfe4ebf2e1b31eab29ac7010a9e9040596159f8732bf8ab9da961bdb"}
Sep 11 00:59:16.525293 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.525229    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jfgl6" event={"ID":"efecf1f1-ec51-4c26-bd4a-56f81bce1f52","Type":"ContainerStarted","Data":"340e61694924caaf329771ef27d40cbaa9ba788b285be6f469ff15ed1eaafa5d"}
Sep 11 00:59:16.525293 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.525258    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jfgl6" event={"ID":"efecf1f1-ec51-4c26-bd4a-56f81bce1f52","Type":"ContainerStarted","Data":"e4beaa86853e69f32f7216b585cf6c71e5e9e428fcee5a5dd473e26c0b07a495"}
Sep 11 00:59:16.527056 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.527032    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qkfrr" event={"ID":"4113c6a8-603c-4e15-a001-047284c67098","Type":"ContainerStarted","Data":"f383ebd65b7bc947b09115e4719146f1ac084d08f607bf3298c33f22bb368c07"}
Sep 11 00:59:16.527164 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.527078    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qkfrr" event={"ID":"4113c6a8-603c-4e15-a001-047284c67098","Type":"ContainerStarted","Data":"d68fcef758f79be3bcac264531637613893b2c5f21a4b4844582359e5806bee1"}
Sep 11 00:59:16.527164 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.527131    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:59:16.528756 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.528734    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gg5rc" event={"ID":"2484980c-48af-426f-b3c2-95afc18ed2a2","Type":"ContainerStarted","Data":"d07dcf8227a9c92a618e7b027eaa1805e4b4ed46329ce7ed3208c65a2bbb4e4f"}
Sep 11 00:59:16.536428 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.536382    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cd56bb784-gtbv6" podStartSLOduration=2.2199017 podStartE2EDuration="8.536369451s" totalImagesPullingTime="6.316467751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:16.535258256 +0000 UTC m=+76.049858439" watchObservedRunningTime="2026-09-11 00:59:16.536369451 +0000 UTC m=+76.050969660"
Sep 11 00:59:16.554230 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.554178    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qkfrr" podStartSLOduration=33.951590317 podStartE2EDuration="41.554162925s" totalImagesPullingTime="7.602572608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:16.55391771 +0000 UTC m=+76.068517907" watchObservedRunningTime="2026-09-11 00:59:16.554162925 +0000 UTC m=+76.068763102"
Sep 11 00:59:16.568199 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.568151    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gg5rc" podStartSLOduration=67.266984284 podStartE2EDuration="1m15.568134499s" totalImagesPullingTime="8.301150215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:16.567649351 +0000 UTC m=+76.082249534" watchObservedRunningTime="2026-09-11 00:59:16.568134499 +0000 UTC m=+76.082734683"
Sep 11 00:59:16.581990 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.581935    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v55tj" podStartSLOduration=67.593673401 podStartE2EDuration="1m15.581920688s" totalImagesPullingTime="7.988247287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:16.580774713 +0000 UTC m=+76.095374907" watchObservedRunningTime="2026-09-11 00:59:16.581920688 +0000 UTC m=+76.096520896"
Sep 11 00:59:16.597413 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.597352    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-bn4l7" podStartSLOduration=2.848582738 podStartE2EDuration="8.597331888s" totalImagesPullingTime="5.74874915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:16.596088188 +0000 UTC m=+76.110688376" watchObservedRunningTime="2026-09-11 00:59:16.597331888 +0000 UTC m=+76.111932072"
Sep 11 00:59:16.615700 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:16.615653    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-784ffbcd54-ccvhc" podStartSLOduration=6.615636823 podStartE2EDuration="6.615636823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:16.614881543 +0000 UTC m=+76.129481725" watchObservedRunningTime="2026-09-11 00:59:16.615636823 +0000 UTC m=+76.130237039"
Sep 11 00:59:18.618880 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.618838    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:18.619423 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.618932    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:18.624134 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.624110    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:18.798842 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.798805    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kc2t5"]
Sep 11 00:59:18.801125 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.801102    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:18.804096 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.804078    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 00:59:18.804831 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.804809    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 00:59:18.804917 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.804844    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 00:59:18.805029 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.805000    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vwq6p\""
Sep 11 00:59:18.809781 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.809761    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"]
Sep 11 00:59:18.811857 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.811841    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:18.814398 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.814328    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 11 00:59:18.815262 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.815243    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 11 00:59:18.815341 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.815262    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 11 00:59:18.815341 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.815320    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-c2xzt\""
Sep 11 00:59:18.833140 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.833067    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"]
Sep 11 00:59:18.902510 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902431    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-wtmp\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:18.902510 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902468    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-tls\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:18.902510 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902499    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4wst\" (UniqueName: \"kubernetes.io/projected/64e628aa-d490-4d30-8d9f-bdf3f6997008-kube-api-access-k4wst\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:18.902789 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902551    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k222\" (UniqueName: \"kubernetes.io/projected/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-api-access-4k222\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:18.902789 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902672    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:18.902789 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902712    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64e628aa-d490-4d30-8d9f-bdf3f6997008-root\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:18.902789 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:18.902789 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902789    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64e628aa-d490-4d30-8d9f-bdf3f6997008-sys\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:18.902984 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902821    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64e628aa-d490-4d30-8d9f-bdf3f6997008-metrics-client-ca\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:18.902984 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902839    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:18.902984 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902877    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-textfile\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:18.902984 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902899    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-accelerators-collector-config\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:18.902984 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902920    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:18.902984 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.902956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e993eca-e4a0-4c53-a347-4a6258649c74-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:18.903276 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:18.903007    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5e993eca-e4a0-4c53-a347-4a6258649c74-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.003998 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.003965    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-textfile\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.003998 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004002    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-accelerators-collector-config\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.004255 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004023    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.004255 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004065    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e993eca-e4a0-4c53-a347-4a6258649c74-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.004255 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004093    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5e993eca-e4a0-4c53-a347-4a6258649c74-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.004255 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-wtmp\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.004255 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-tls\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.004255 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004178    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4wst\" (UniqueName: \"kubernetes.io/projected/64e628aa-d490-4d30-8d9f-bdf3f6997008-kube-api-access-k4wst\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.004255 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k222\" (UniqueName: \"kubernetes.io/projected/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-api-access-4k222\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.004255 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004239    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.004680 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004267    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64e628aa-d490-4d30-8d9f-bdf3f6997008-root\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.004680 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004290    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.004680 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004315    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64e628aa-d490-4d30-8d9f-bdf3f6997008-sys\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.004680 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004342    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64e628aa-d490-4d30-8d9f-bdf3f6997008-metrics-client-ca\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.004680 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004365    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.004680 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-textfile\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.004680 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004449    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64e628aa-d490-4d30-8d9f-bdf3f6997008-root\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.004680 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:59:19.004489    2559 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 11 00:59:19.004680 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:59:19.004548    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-tls podName:5e993eca-e4a0-4c53-a347-4a6258649c74 nodeName:}" failed. No retries permitted until 2026-09-11 00:59:19.504527718 +0000 UTC m=+79.019127892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-p6l84" (UID: "5e993eca-e4a0-4c53-a347-4a6258649c74") : secret "kube-state-metrics-tls" not found
Sep 11 00:59:19.004680 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004557    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5e993eca-e4a0-4c53-a347-4a6258649c74-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.005180 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004861    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64e628aa-d490-4d30-8d9f-bdf3f6997008-sys\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.005180 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004883    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-wtmp\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.005180 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004921    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e993eca-e4a0-4c53-a347-4a6258649c74-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.005180 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.004941    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.005528 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.005484    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-accelerators-collector-config\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.005722 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.005692    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64e628aa-d490-4d30-8d9f-bdf3f6997008-metrics-client-ca\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.007205 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.007178    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.007451 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.007423    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.007577 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.007560    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64e628aa-d490-4d30-8d9f-bdf3f6997008-node-exporter-tls\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.013562 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.013502    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4wst\" (UniqueName: \"kubernetes.io/projected/64e628aa-d490-4d30-8d9f-bdf3f6997008-kube-api-access-k4wst\") pod \"node-exporter-kc2t5\" (UID: \"64e628aa-d490-4d30-8d9f-bdf3f6997008\") " pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.014172 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.014149    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k222\" (UniqueName: \"kubernetes.io/projected/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-api-access-4k222\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.111431 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.111394    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kc2t5"
Sep 11 00:59:19.509963 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.509920    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:19.510148 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:59:19.509941    2559 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 11 00:59:19.510148 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:59:19.510062    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-tls podName:5e993eca-e4a0-4c53-a347-4a6258649c74 nodeName:}" failed. No retries permitted until 2026-09-11 00:59:20.510039801 +0000 UTC m=+80.024639977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-p6l84" (UID: "5e993eca-e4a0-4c53-a347-4a6258649c74") : secret "kube-state-metrics-tls" not found
Sep 11 00:59:19.544703 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.544671    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:19.932984 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.932953    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 00:59:19.938218 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.938194    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:19.940794 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.940769    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 00:59:19.940794 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.940787    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 00:59:19.941060 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.941001    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 00:59:19.941060 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.941052    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 00:59:19.941261 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.941247    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h6mq8\""
Sep 11 00:59:19.941319 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.941260    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 00:59:19.941372 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.941331    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 00:59:19.941372 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.941348    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 00:59:19.941372 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.941353    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 00:59:19.941743 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.941727    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 00:59:19.957436 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:19.957403    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 00:59:20.014027 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.013977    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014027 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014280 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-out\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014280 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014088    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014280 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014280 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014280 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014201    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014280 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014230    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014280 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014258    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014653 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014291    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014653 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014336    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-web-config\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014653 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014422    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.014653 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.014468    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8kz6\" (UniqueName: \"kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-kube-api-access-x8kz6\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115096 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115064    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115297 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115101    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115297 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115134    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115297 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-web-config\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115474 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115415    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115474 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115467    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8kz6\" (UniqueName: \"kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-kube-api-access-x8kz6\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115571 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115496    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115668 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115643    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115733 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115692    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-out\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115733 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115726    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115840 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115749    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115840 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115791    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115940 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.115940 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.115920    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.116799 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.116704    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.118176 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.118149    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.118176 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.118169    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-web-config\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.118361 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.118312    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.118649 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.118564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.119802 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.119707    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.120099 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.120077    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.120178 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.120109    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.120277 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.120260    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.120996 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.120976    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-out\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.121220 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.121199    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.128483 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.128440    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8kz6\" (UniqueName: \"kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-kube-api-access-x8kz6\") pod \"alertmanager-main-0\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.249640 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.249549    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 00:59:20.519619 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.519579    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:20.522258 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.522224    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e993eca-e4a0-4c53-a347-4a6258649c74-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-p6l84\" (UID: \"5e993eca-e4a0-4c53-a347-4a6258649c74\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:20.621319 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.621291    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"
Sep 11 00:59:20.991760 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.991667    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:20.991760 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.991748    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:20.997066 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:20.997022    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:21.552562 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.552532    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 00:59:21.596633 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.596588    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd56bb784-gtbv6"]
Sep 11 00:59:21.826330 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.826245    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-79f995d456-4zrm9"]
Sep 11 00:59:21.831165 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.831138    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:21.834224 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.834199    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 00:59:21.834509 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.834489    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 00:59:21.834887 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.834833    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-k5z7g\""
Sep 11 00:59:21.835842 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.835822    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bjhqdi0mf10i6\""
Sep 11 00:59:21.836089 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.836069    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 00:59:21.836190 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.836172    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 00:59:21.836701 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.836674    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 00:59:21.843541 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.843460    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-79f995d456-4zrm9"]
Sep 11 00:59:21.931045 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.931012    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-tls\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:21.931243 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.931063    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:21.931243 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.931104    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:21.931243 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.931169    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/676ff702-be28-4f99-9d85-b000a9af43ed-metrics-client-ca\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:21.931243 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.931207    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcnrq\" (UniqueName: \"kubernetes.io/projected/676ff702-be28-4f99-9d85-b000a9af43ed-kube-api-access-vcnrq\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:21.931466 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.931251    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:21.931466 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.931277    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-grpc-tls\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:21.931466 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:21.931334    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.032651 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.032588    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.033122 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.032675    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-tls\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.033122 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.032713    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.033122 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.032742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.033122 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.032771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/676ff702-be28-4f99-9d85-b000a9af43ed-metrics-client-ca\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.033352 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.033208    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcnrq\" (UniqueName: \"kubernetes.io/projected/676ff702-be28-4f99-9d85-b000a9af43ed-kube-api-access-vcnrq\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.033352 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.033274    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.033352 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.033303    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-grpc-tls\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.033667 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.033644    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/676ff702-be28-4f99-9d85-b000a9af43ed-metrics-client-ca\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.035910 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.035775    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.035910 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.035806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.036250 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.036229    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-tls\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.036327 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.036279    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.036398 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.036355    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-grpc-tls\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.036709 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.036686    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/676ff702-be28-4f99-9d85-b000a9af43ed-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.042040 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.042018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcnrq\" (UniqueName: \"kubernetes.io/projected/676ff702-be28-4f99-9d85-b000a9af43ed-kube-api-access-vcnrq\") pod \"thanos-querier-79f995d456-4zrm9\" (UID: \"676ff702-be28-4f99-9d85-b000a9af43ed\") " pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:22.143451 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:22.143358    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:23.602891 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.602756    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"]
Sep 11 00:59:23.606250 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.606225    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"
Sep 11 00:59:23.609772 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.609745    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xprsq\""
Sep 11 00:59:23.609910 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.609780    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 11 00:59:23.617801 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.616386    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"]
Sep 11 00:59:23.748219 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.748179    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rm22\" (UniqueName: \"kubernetes.io/projected/8b51c10d-691c-4769-a8cc-55d8f726b9a0-kube-api-access-7rm22\") pod \"monitoring-plugin-5d85778b9d-rqrlm\" (UID: \"8b51c10d-691c-4769-a8cc-55d8f726b9a0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"
Sep 11 00:59:23.748436 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.748275    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8b51c10d-691c-4769-a8cc-55d8f726b9a0-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-rqrlm\" (UID: \"8b51c10d-691c-4769-a8cc-55d8f726b9a0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"
Sep 11 00:59:23.848949 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.848908    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8b51c10d-691c-4769-a8cc-55d8f726b9a0-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-rqrlm\" (UID: \"8b51c10d-691c-4769-a8cc-55d8f726b9a0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"
Sep 11 00:59:23.849129 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.849000    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rm22\" (UniqueName: \"kubernetes.io/projected/8b51c10d-691c-4769-a8cc-55d8f726b9a0-kube-api-access-7rm22\") pod \"monitoring-plugin-5d85778b9d-rqrlm\" (UID: \"8b51c10d-691c-4769-a8cc-55d8f726b9a0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"
Sep 11 00:59:23.852029 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.851995    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8b51c10d-691c-4769-a8cc-55d8f726b9a0-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-rqrlm\" (UID: \"8b51c10d-691c-4769-a8cc-55d8f726b9a0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"
Sep 11 00:59:23.861620 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.861530    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rm22\" (UniqueName: \"kubernetes.io/projected/8b51c10d-691c-4769-a8cc-55d8f726b9a0-kube-api-access-7rm22\") pod \"monitoring-plugin-5d85778b9d-rqrlm\" (UID: \"8b51c10d-691c-4769-a8cc-55d8f726b9a0\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"
Sep 11 00:59:23.918456 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:23.918420    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"
Sep 11 00:59:24.917341 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:24.917304    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fc877f5b-sv4xg"]
Sep 11 00:59:24.927414 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:24.927382    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:24.934412 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:24.934382    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fc877f5b-sv4xg"]
Sep 11 00:59:25.061167 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.060704    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj5wn\" (UniqueName: \"kubernetes.io/projected/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-kube-api-access-hj5wn\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.061167 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.060757    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-service-ca\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.061167 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.060835    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-serving-cert\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.061167 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.060869    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-oauth-config\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.061167 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.060908    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-config\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.061167 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.060954    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-oauth-serving-cert\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.061167 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.060981    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-trusted-ca-bundle\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.162507 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.162472    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-service-ca\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.162816 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.162798    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-serving-cert\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.162925 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.162912    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-oauth-config\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.163036 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.163022    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-config\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.163384 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.163103    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-service-ca\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.163384 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.163141    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-oauth-serving-cert\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.163384 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.163173    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-trusted-ca-bundle\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.163939 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.163898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-config\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.164043 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.164000    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-oauth-serving-cert\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.164043 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.164029    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj5wn\" (UniqueName: \"kubernetes.io/projected/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-kube-api-access-hj5wn\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.165393 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.164445    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-trusted-ca-bundle\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.166533 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.166470    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-oauth-config\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.167903 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.167880    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-serving-cert\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.172416 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.172370    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj5wn\" (UniqueName: \"kubernetes.io/projected/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-kube-api-access-hj5wn\") pod \"console-fc877f5b-sv4xg\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") " pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.236970 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.236925    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:25.315950 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.314181    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-79f995d456-4zrm9"]
Sep 11 00:59:25.317566 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:25.317506    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod676ff702_be28_4f99_9d85_b000a9af43ed.slice/crio-a3547e41a776c2bece650f63342f8bae28b557379f4c5aa3da90b255fdb647c7 WatchSource:0}: Error finding container a3547e41a776c2bece650f63342f8bae28b557379f4c5aa3da90b255fdb647c7: Status 404 returned error can't find the container with id a3547e41a776c2bece650f63342f8bae28b557379f4c5aa3da90b255fdb647c7
Sep 11 00:59:25.324682 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.324654    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"]
Sep 11 00:59:25.332026 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.332002    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84"]
Sep 11 00:59:25.335066 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:25.335036    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e993eca_e4a0_4c53_a347_4a6258649c74.slice/crio-f450ff15f218a25e2a4396790a5d4ee97dde9315231f060c34c9378db7cde21b WatchSource:0}: Error finding container f450ff15f218a25e2a4396790a5d4ee97dde9315231f060c34c9378db7cde21b: Status 404 returned error can't find the container with id f450ff15f218a25e2a4396790a5d4ee97dde9315231f060c34c9378db7cde21b
Sep 11 00:59:25.365142 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.365110    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 00:59:25.368585 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:25.368543    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee9dc224_ece0_430f_a967_f35905cbdc1a.slice/crio-1567ebacc8aa0f9bd1b5b380b7d0ed704f526f9eb3158f88a9ebca85b62f6bb5 WatchSource:0}: Error finding container 1567ebacc8aa0f9bd1b5b380b7d0ed704f526f9eb3158f88a9ebca85b62f6bb5: Status 404 returned error can't find the container with id 1567ebacc8aa0f9bd1b5b380b7d0ed704f526f9eb3158f88a9ebca85b62f6bb5
Sep 11 00:59:25.525066 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.525000    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fc877f5b-sv4xg"]
Sep 11 00:59:25.531156 ip-10-0-138-14 kubenswrapper[2559]: W0911 00:59:25.531122    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef27599_ba2c_48a7_a7ef_2ef35ee8254c.slice/crio-ff5cc0cb8f699a4fae15c59665ef011b046f7fef2be4f94790e9d1636926d721 WatchSource:0}: Error finding container ff5cc0cb8f699a4fae15c59665ef011b046f7fef2be4f94790e9d1636926d721: Status 404 returned error can't find the container with id ff5cc0cb8f699a4fae15c59665ef011b046f7fef2be4f94790e9d1636926d721
Sep 11 00:59:25.573357 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.572441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9" event={"ID":"676ff702-be28-4f99-9d85-b000a9af43ed","Type":"ContainerStarted","Data":"a3547e41a776c2bece650f63342f8bae28b557379f4c5aa3da90b255fdb647c7"}
Sep 11 00:59:25.581201 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.581119    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jfgl6" event={"ID":"efecf1f1-ec51-4c26-bd4a-56f81bce1f52","Type":"ContainerStarted","Data":"51492633ba960302a675a6dcf07c77741b1c7c5d68755eb4501df4aebfc6c74d"}
Sep 11 00:59:25.583007 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.582967    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc877f5b-sv4xg" event={"ID":"cef27599-ba2c-48a7-a7ef-2ef35ee8254c","Type":"ContainerStarted","Data":"ff5cc0cb8f699a4fae15c59665ef011b046f7fef2be4f94790e9d1636926d721"}
Sep 11 00:59:25.587425 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.587333    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerStarted","Data":"1567ebacc8aa0f9bd1b5b380b7d0ed704f526f9eb3158f88a9ebca85b62f6bb5"}
Sep 11 00:59:25.589960 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.589930    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84" event={"ID":"5e993eca-e4a0-4c53-a347-4a6258649c74","Type":"ContainerStarted","Data":"f450ff15f218a25e2a4396790a5d4ee97dde9315231f060c34c9378db7cde21b"}
Sep 11 00:59:25.592413 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.592364    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm" event={"ID":"8b51c10d-691c-4769-a8cc-55d8f726b9a0","Type":"ContainerStarted","Data":"1fea573c24eaa2ee6ef60815de1d5f6cb1cb80d5dbf183f1ce52aa59f3f5ff88"}
Sep 11 00:59:25.598036 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.598002    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-p5cwr" event={"ID":"b8c6b379-1a03-4d54-ad83-ae2217a4df4f","Type":"ContainerStarted","Data":"73f826155a7a73e24f413b37265c2a38190c87dd9d7888f3281d710b84463489"}
Sep 11 00:59:25.598634 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.598595    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-p5cwr"
Sep 11 00:59:25.601561 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.601528    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kc2t5" event={"ID":"64e628aa-d490-4d30-8d9f-bdf3f6997008","Type":"ContainerStarted","Data":"1ad33555d0d7595a4b9cd387daaf1243baa90643a74d99fe1f38674b66b9224f"}
Sep 11 00:59:25.605832 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.605783    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jfgl6" podStartSLOduration=34.493851654 podStartE2EDuration="43.605763246s" totalImagesPullingTime="9.111911592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:58:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:25.60552972 +0000 UTC m=+85.120129905" watchObservedRunningTime="2026-09-11 00:59:25.605763246 +0000 UTC m=+85.120363430"
Sep 11 00:59:25.607524 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.607503    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-p5cwr"
Sep 11 00:59:25.632139 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:25.631124    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-p5cwr" podStartSLOduration=1.270511322 podStartE2EDuration="20.631102757s" totalImagesPullingTime="19.360591435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:25.630742931 +0000 UTC m=+85.145343115" watchObservedRunningTime="2026-09-11 00:59:25.631102757 +0000 UTC m=+85.145702942"
Sep 11 00:59:26.537570 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:26.537466    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qkfrr"
Sep 11 00:59:26.620186 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:26.620095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc877f5b-sv4xg" event={"ID":"cef27599-ba2c-48a7-a7ef-2ef35ee8254c","Type":"ContainerStarted","Data":"5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60"}
Sep 11 00:59:26.645830 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:26.644102    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fc877f5b-sv4xg" podStartSLOduration=2.6440779279999997 podStartE2EDuration="2.644077928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:26.64394355 +0000 UTC m=+86.158543734" watchObservedRunningTime="2026-09-11 00:59:26.644077928 +0000 UTC m=+86.158678114"
Sep 11 00:59:30.639387 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.639296    2559 generic.go:356] "Generic (PLEG): container finished" podID="64e628aa-d490-4d30-8d9f-bdf3f6997008" containerID="43681e18cd353602070059499fb0701f6965edfc3861be8ae698becab9abe83b" exitCode=0
Sep 11 00:59:30.639875 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.639382    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kc2t5" event={"ID":"64e628aa-d490-4d30-8d9f-bdf3f6997008","Type":"ContainerDied","Data":"43681e18cd353602070059499fb0701f6965edfc3861be8ae698becab9abe83b"}
Sep 11 00:59:30.642311 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.642280    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9" event={"ID":"676ff702-be28-4f99-9d85-b000a9af43ed","Type":"ContainerStarted","Data":"4283f4f245b93552ef13cd5afb115275c080d1283eb5d3f2071a644c3ff99419"}
Sep 11 00:59:30.642430 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.642362    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9" event={"ID":"676ff702-be28-4f99-9d85-b000a9af43ed","Type":"ContainerStarted","Data":"c68a7dbaad826371d392312b29668fa8160627e66c6942faa37c308d0be8ab50"}
Sep 11 00:59:30.642430 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.642406    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9" event={"ID":"676ff702-be28-4f99-9d85-b000a9af43ed","Type":"ContainerStarted","Data":"1e21d4697416ed01d515267e0f205c52a27ec30d45dac4471a5310fc9cca5098"}
Sep 11 00:59:30.644260 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.644235    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerID="ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd" exitCode=0
Sep 11 00:59:30.644370 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.644317    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerDied","Data":"ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd"}
Sep 11 00:59:30.647476 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.647454    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84" event={"ID":"5e993eca-e4a0-4c53-a347-4a6258649c74","Type":"ContainerStarted","Data":"05415ab11fdfd838966ff64c1cb630c0974305f96cfc9d214bed0e323d5b2275"}
Sep 11 00:59:30.647571 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.647500    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84" event={"ID":"5e993eca-e4a0-4c53-a347-4a6258649c74","Type":"ContainerStarted","Data":"719b3b495c0ef7c36bfa3d31ba75deb5f0074b3eb892c9f92f610dc46fd6c3a9"}
Sep 11 00:59:30.647571 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.647530    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84" event={"ID":"5e993eca-e4a0-4c53-a347-4a6258649c74","Type":"ContainerStarted","Data":"f99d386dc76464bc0fe8816c2d9c28189b67bb12599307bef1c38482ed21e9fe"}
Sep 11 00:59:30.649297 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.649271    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm" event={"ID":"8b51c10d-691c-4769-a8cc-55d8f726b9a0","Type":"ContainerStarted","Data":"9058cfc6599376878a0df5c225528689275e0fc512d6b65ab310a5b8079889e2"}
Sep 11 00:59:30.649520 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.649476    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"
Sep 11 00:59:30.655488 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.655465    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm"
Sep 11 00:59:30.706500 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.706445    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-p6l84" podStartSLOduration=8.265319533 podStartE2EDuration="12.706423521s" totalImagesPullingTime="4.441103988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:30.704868623 +0000 UTC m=+90.219468829" watchObservedRunningTime="2026-09-11 00:59:30.706423521 +0000 UTC m=+90.221023706"
Sep 11 00:59:30.722847 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:30.722800    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-rqrlm" podStartSLOduration=3.274391252 podStartE2EDuration="7.722782837s" totalImagesPullingTime="4.448391585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:30.72139186 +0000 UTC m=+90.235992058" watchObservedRunningTime="2026-09-11 00:59:30.722782837 +0000 UTC m=+90.237383020"
Sep 11 00:59:31.659320 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:31.659279    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kc2t5" event={"ID":"64e628aa-d490-4d30-8d9f-bdf3f6997008","Type":"ContainerStarted","Data":"e754c2d61eff56b8d49f85e44d6040001ae28ab524234ca5d9b5836069af66b4"}
Sep 11 00:59:31.660414 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:31.659349    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kc2t5" event={"ID":"64e628aa-d490-4d30-8d9f-bdf3f6997008","Type":"ContainerStarted","Data":"32a59b29969b3d9efca29bf8273aaeb7205b82e26e82ec3bb2460d8181d03356"}
Sep 11 00:59:31.682627 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:31.682523    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kc2t5" podStartSLOduration=8.818883485 podStartE2EDuration="13.682503489s" totalImagesPullingTime="4.761207313s" totalInitContainerRuntime="102.412691ms" isStatefulPod=true podCreationTimestamp="2026-09-11 00:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:31.680433008 +0000 UTC m=+91.195033187" watchObservedRunningTime="2026-09-11 00:59:31.682503489 +0000 UTC m=+91.197103673"
Sep 11 00:59:32.668243 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:32.668206    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9" event={"ID":"676ff702-be28-4f99-9d85-b000a9af43ed","Type":"ContainerStarted","Data":"460e2eba88288ae730ac06f67af734b0d9f6510c752e596ff4dfb1762e30e167"}
Sep 11 00:59:33.677819 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.677765    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerStarted","Data":"6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1"}
Sep 11 00:59:33.678266 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.677857    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerStarted","Data":"13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431"}
Sep 11 00:59:33.678266 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.677901    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerStarted","Data":"2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5"}
Sep 11 00:59:33.678266 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.677945    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerStarted","Data":"d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3"}
Sep 11 00:59:33.678266 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.677981    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerStarted","Data":"d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e"}
Sep 11 00:59:33.678266 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.678021    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerStarted","Data":"1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a"}
Sep 11 00:59:33.681561 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.681525    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9" event={"ID":"676ff702-be28-4f99-9d85-b000a9af43ed","Type":"ContainerStarted","Data":"3acb1f4112afb6b9112941386a240b59d16dfae1737ef3a564a95ec08821d05b"}
Sep 11 00:59:33.681711 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.681621    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9" event={"ID":"676ff702-be28-4f99-9d85-b000a9af43ed","Type":"ContainerStarted","Data":"8ce3010ade8b1ce3158bcc5097582904e05bdeeff72c55659dbb496acd7bd18e"}
Sep 11 00:59:33.681831 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.681811    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:33.704948 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.704893    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.074073086 podStartE2EDuration="14.704869509s" totalImagesPullingTime="6.607315976s" totalInitContainerRuntime="23.480447ms" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:33.702201575 +0000 UTC m=+93.216801782" watchObservedRunningTime="2026-09-11 00:59:33.704869509 +0000 UTC m=+93.219469694"
Sep 11 00:59:33.730078 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:33.729954    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9" podStartSLOduration=5.934868505 podStartE2EDuration="12.729928363s" totalImagesPullingTime="6.795059858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:59:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:59:33.727887193 +0000 UTC m=+93.242487401" watchObservedRunningTime="2026-09-11 00:59:33.729928363 +0000 UTC m=+93.244528550"
Sep 11 00:59:35.237649 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:35.237598    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:35.238087 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:35.237684    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:35.242159 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:35.242138    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:35.693359 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:35.693331    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 00:59:35.745404 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:35.745370    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-784ffbcd54-ccvhc"]
Sep 11 00:59:39.691828 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:39.691803    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-79f995d456-4zrm9"
Sep 11 00:59:46.622419 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:46.622381    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cd56bb784-gtbv6" podUID="6b8ce73c-a602-4633-9a36-88482852b2cf" containerName="console" containerID="cri-o://592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229" gracePeriod=15
Sep 11 00:59:46.890310 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:46.890238    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:47.071071 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.071032    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-oauth-config\") pod \"6b8ce73c-a602-4633-9a36-88482852b2cf\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") "
Sep 11 00:59:47.071316 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.071098    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-console-config\") pod \"6b8ce73c-a602-4633-9a36-88482852b2cf\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") "
Sep 11 00:59:47.071316 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.071135    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-oauth-serving-cert\") pod \"6b8ce73c-a602-4633-9a36-88482852b2cf\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") "
Sep 11 00:59:47.071316 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.071167    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nhp6\" (UniqueName: \"kubernetes.io/projected/6b8ce73c-a602-4633-9a36-88482852b2cf-kube-api-access-7nhp6\") pod \"6b8ce73c-a602-4633-9a36-88482852b2cf\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") "
Sep 11 00:59:47.071316 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.071214    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-service-ca\") pod \"6b8ce73c-a602-4633-9a36-88482852b2cf\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") "
Sep 11 00:59:47.071316 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.071239    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-serving-cert\") pod \"6b8ce73c-a602-4633-9a36-88482852b2cf\" (UID: \"6b8ce73c-a602-4633-9a36-88482852b2cf\") "
Sep 11 00:59:47.071564 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.071483    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-console-config" (OuterVolumeSpecName: "console-config") pod "6b8ce73c-a602-4633-9a36-88482852b2cf" (UID: "6b8ce73c-a602-4633-9a36-88482852b2cf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:59:47.071564 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.071490    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6b8ce73c-a602-4633-9a36-88482852b2cf" (UID: "6b8ce73c-a602-4633-9a36-88482852b2cf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:59:47.071701 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.071557    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-service-ca" (OuterVolumeSpecName: "service-ca") pod "6b8ce73c-a602-4633-9a36-88482852b2cf" (UID: "6b8ce73c-a602-4633-9a36-88482852b2cf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:59:47.073567 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.073533    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6b8ce73c-a602-4633-9a36-88482852b2cf" (UID: "6b8ce73c-a602-4633-9a36-88482852b2cf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:59:47.073694 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.073630    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b8ce73c-a602-4633-9a36-88482852b2cf-kube-api-access-7nhp6" (OuterVolumeSpecName: "kube-api-access-7nhp6") pod "6b8ce73c-a602-4633-9a36-88482852b2cf" (UID: "6b8ce73c-a602-4633-9a36-88482852b2cf"). InnerVolumeSpecName "kube-api-access-7nhp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:59:47.073694 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.073641    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6b8ce73c-a602-4633-9a36-88482852b2cf" (UID: "6b8ce73c-a602-4633-9a36-88482852b2cf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:59:47.172585 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.172503    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-serving-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:47.172585 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.172533    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b8ce73c-a602-4633-9a36-88482852b2cf-console-oauth-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:47.172585 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.172542    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-console-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:47.172585 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.172551    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-oauth-serving-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:47.172585 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.172559    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nhp6\" (UniqueName: \"kubernetes.io/projected/6b8ce73c-a602-4633-9a36-88482852b2cf-kube-api-access-7nhp6\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:47.172585 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.172568    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ce73c-a602-4633-9a36-88482852b2cf-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 00:59:47.536984 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.536959    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v55tj"
Sep 11 00:59:47.742415 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.742385    2559 generic.go:356] "Generic (PLEG): container finished" podID="6b8ce73c-a602-4633-9a36-88482852b2cf" containerID="592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229" exitCode=0
Sep 11 00:59:47.742949 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.742447    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd56bb784-gtbv6"
Sep 11 00:59:47.742949 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.742475    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd56bb784-gtbv6" event={"ID":"6b8ce73c-a602-4633-9a36-88482852b2cf","Type":"ContainerDied","Data":"592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229"}
Sep 11 00:59:47.742949 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.742523    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd56bb784-gtbv6" event={"ID":"6b8ce73c-a602-4633-9a36-88482852b2cf","Type":"ContainerDied","Data":"89231431d2b01a7b0f829d8a7719cb15de088aff42167abfe5520c2ad83b0734"}
Sep 11 00:59:47.742949 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.742557    2559 scope.go:117] "RemoveContainer" containerID="592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229"
Sep 11 00:59:47.751053 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.751036    2559 scope.go:117] "RemoveContainer" containerID="592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229"
Sep 11 00:59:47.751328 ip-10-0-138-14 kubenswrapper[2559]: E0911 00:59:47.751302    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229\": container with ID starting with 592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229 not found: ID does not exist" containerID="592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229"
Sep 11 00:59:47.751390 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.751328    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229"} err="failed to get container status \"592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229\": rpc error: code = NotFound desc = could not find container \"592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229\": container with ID starting with 592ce1518a94c7c021ea1d41ae201c380ff3f50db46c0df80f2dc60a168b7229 not found: ID does not exist"
Sep 11 00:59:47.759501 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.759478    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd56bb784-gtbv6"]
Sep 11 00:59:47.762951 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:47.762930    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cd56bb784-gtbv6"]
Sep 11 00:59:49.108827 ip-10-0-138-14 kubenswrapper[2559]: I0911 00:59:49.108796    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b8ce73c-a602-4633-9a36-88482852b2cf" path="/var/lib/kubelet/pods/6b8ce73c-a602-4633-9a36-88482852b2cf/volumes"
Sep 11 01:00:00.769376 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:00.769329    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-784ffbcd54-ccvhc" podUID="0481f0ef-71c9-4dae-86ff-13510383e309" containerName="console" containerID="cri-o://e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8" gracePeriod=15
Sep 11 01:00:01.043228 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.043200    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 01:00:01.074424 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.074398    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-oauth-config\") pod \"0481f0ef-71c9-4dae-86ff-13510383e309\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") "
Sep 11 01:00:01.074599 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.074440    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfxm8\" (UniqueName: \"kubernetes.io/projected/0481f0ef-71c9-4dae-86ff-13510383e309-kube-api-access-zfxm8\") pod \"0481f0ef-71c9-4dae-86ff-13510383e309\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") "
Sep 11 01:00:01.074599 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.074465    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-serving-cert\") pod \"0481f0ef-71c9-4dae-86ff-13510383e309\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") "
Sep 11 01:00:01.074599 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.074488    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-console-config\") pod \"0481f0ef-71c9-4dae-86ff-13510383e309\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") "
Sep 11 01:00:01.074599 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.074526    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-service-ca\") pod \"0481f0ef-71c9-4dae-86ff-13510383e309\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") "
Sep 11 01:00:01.074599 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.074556    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-oauth-serving-cert\") pod \"0481f0ef-71c9-4dae-86ff-13510383e309\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") "
Sep 11 01:00:01.074599 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.074581    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-trusted-ca-bundle\") pod \"0481f0ef-71c9-4dae-86ff-13510383e309\" (UID: \"0481f0ef-71c9-4dae-86ff-13510383e309\") "
Sep 11 01:00:01.075066 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.075019    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-console-config" (OuterVolumeSpecName: "console-config") pod "0481f0ef-71c9-4dae-86ff-13510383e309" (UID: "0481f0ef-71c9-4dae-86ff-13510383e309"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:01.075191 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.075135    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0481f0ef-71c9-4dae-86ff-13510383e309" (UID: "0481f0ef-71c9-4dae-86ff-13510383e309"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:01.075191 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.075168    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-service-ca" (OuterVolumeSpecName: "service-ca") pod "0481f0ef-71c9-4dae-86ff-13510383e309" (UID: "0481f0ef-71c9-4dae-86ff-13510383e309"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:01.075191 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.075176    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0481f0ef-71c9-4dae-86ff-13510383e309" (UID: "0481f0ef-71c9-4dae-86ff-13510383e309"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:01.076758 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.076727    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0481f0ef-71c9-4dae-86ff-13510383e309" (UID: "0481f0ef-71c9-4dae-86ff-13510383e309"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:01.076873 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.076815    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0481f0ef-71c9-4dae-86ff-13510383e309" (UID: "0481f0ef-71c9-4dae-86ff-13510383e309"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:01.076873 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.076861    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0481f0ef-71c9-4dae-86ff-13510383e309-kube-api-access-zfxm8" (OuterVolumeSpecName: "kube-api-access-zfxm8") pod "0481f0ef-71c9-4dae-86ff-13510383e309" (UID: "0481f0ef-71c9-4dae-86ff-13510383e309"). InnerVolumeSpecName "kube-api-access-zfxm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:00:01.176088 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.176049    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfxm8\" (UniqueName: \"kubernetes.io/projected/0481f0ef-71c9-4dae-86ff-13510383e309-kube-api-access-zfxm8\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:01.176088 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.176078    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-serving-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:01.176088 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.176088    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-console-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:01.176088 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.176097    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:01.176088 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.176104    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-oauth-serving-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:01.176374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.176112    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0481f0ef-71c9-4dae-86ff-13510383e309-trusted-ca-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:01.176374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.176120    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0481f0ef-71c9-4dae-86ff-13510383e309-console-oauth-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:01.801448 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.801412    2559 generic.go:356] "Generic (PLEG): container finished" podID="0481f0ef-71c9-4dae-86ff-13510383e309" containerID="e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8" exitCode=0
Sep 11 01:00:01.801946 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.801458    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-784ffbcd54-ccvhc"
Sep 11 01:00:01.801946 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.801494    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784ffbcd54-ccvhc" event={"ID":"0481f0ef-71c9-4dae-86ff-13510383e309","Type":"ContainerDied","Data":"e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8"}
Sep 11 01:00:01.801946 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.801555    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-784ffbcd54-ccvhc" event={"ID":"0481f0ef-71c9-4dae-86ff-13510383e309","Type":"ContainerDied","Data":"386160f8a5cb463610956deed9e176f049d271e19e32f893cd59e374198b959b"}
Sep 11 01:00:01.801946 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.801594    2559 scope.go:117] "RemoveContainer" containerID="e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8"
Sep 11 01:00:01.810137 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.810116    2559 scope.go:117] "RemoveContainer" containerID="e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8"
Sep 11 01:00:01.813141 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:00:01.813114    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8\": container with ID starting with e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8 not found: ID does not exist" containerID="e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8"
Sep 11 01:00:01.813234 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.813148    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8"} err="failed to get container status \"e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8\": rpc error: code = NotFound desc = could not find container \"e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8\": container with ID starting with e4fc88ac5ffcc21593345fc64cd3f9728b5abcd7a6e8d9ba35d9ef171053cab8 not found: ID does not exist"
Sep 11 01:00:01.819153 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.819128    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-784ffbcd54-ccvhc"]
Sep 11 01:00:01.822832 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:01.822812    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-784ffbcd54-ccvhc"]
Sep 11 01:00:03.108471 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:03.108436    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0481f0ef-71c9-4dae-86ff-13510383e309" path="/var/lib/kubelet/pods/0481f0ef-71c9-4dae-86ff-13510383e309/volumes"
Sep 11 01:00:39.148786 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.148747    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 01:00:39.149299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.149043    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="alertmanager" containerID="cri-o://1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a" gracePeriod=120
Sep 11 01:00:39.149299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.149098    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy-metric" containerID="cri-o://13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431" gracePeriod=120
Sep 11 01:00:39.149299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.149158    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="prom-label-proxy" containerID="cri-o://6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1" gracePeriod=120
Sep 11 01:00:39.149299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.149193    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy-web" containerID="cri-o://d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3" gracePeriod=120
Sep 11 01:00:39.149299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.149218    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="config-reloader" containerID="cri-o://d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e" gracePeriod=120
Sep 11 01:00:39.149299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.149193    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy" containerID="cri-o://2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5" gracePeriod=120
Sep 11 01:00:39.953567 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.953533    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerID="6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1" exitCode=0
Sep 11 01:00:39.953567 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.953561    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerID="13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431" exitCode=0
Sep 11 01:00:39.953567 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.953572    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerID="2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5" exitCode=0
Sep 11 01:00:39.953811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.953579    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerID="d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e" exitCode=0
Sep 11 01:00:39.953811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.953586    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerID="1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a" exitCode=0
Sep 11 01:00:39.953811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.953611    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerDied","Data":"6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1"}
Sep 11 01:00:39.953811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.953672    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerDied","Data":"13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431"}
Sep 11 01:00:39.953811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.953700    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerDied","Data":"2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5"}
Sep 11 01:00:39.953811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.953725    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerDied","Data":"d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e"}
Sep 11 01:00:39.953811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:39.953748    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerDied","Data":"1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a"}
Sep 11 01:00:40.386648 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.386625    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:40.494373 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494295    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-metrics-client-ca\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494373 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494335    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-web-config\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494546 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494380    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8kz6\" (UniqueName: \"kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-kube-api-access-x8kz6\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494546 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494482    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-out\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494546 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494523    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-main-db\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494707 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494550    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-web\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494707 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494583    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-cluster-tls-config\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494707 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494648    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-main-tls\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494707 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494681    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-volume\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494915 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494745    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-tls-assets\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494915 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494774    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494915 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494813    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-trusted-ca-bundle\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.494915 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494830    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:40.494915 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494839    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ee9dc224-ece0-430f-a967-f35905cbdc1a\" (UID: \"ee9dc224-ece0-430f-a967-f35905cbdc1a\") "
Sep 11 01:00:40.495163 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.495120    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-metrics-client-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.495163 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.494853    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:00:40.496140 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.496108    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:00:40.497466 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.497426    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-out" (OuterVolumeSpecName: "config-out") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:00:40.497914 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.497893    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-kube-api-access-x8kz6" (OuterVolumeSpecName: "kube-api-access-x8kz6") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "kube-api-access-x8kz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:00:40.498010 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.497909    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:40.498111 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.498086    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:00:40.498187 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.498091    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:40.498428 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.498380    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:40.498596 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.498577    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-volume" (OuterVolumeSpecName: "config-volume") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:40.499079 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.499061    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:40.501572 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.501549    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:40.507512 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.507489    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-web-config" (OuterVolumeSpecName: "web-config") pod "ee9dc224-ece0-430f-a967-f35905cbdc1a" (UID: "ee9dc224-ece0-430f-a967-f35905cbdc1a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:00:40.595533 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595498    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-web-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595533 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595530    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8kz6\" (UniqueName: \"kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-kube-api-access-x8kz6\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595545    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-out\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595555    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-main-db\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595566    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595575    2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-cluster-tls-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595584    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-main-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595593    2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-config-volume\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595622    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee9dc224-ece0-430f-a967-f35905cbdc1a-tls-assets\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595631    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595641    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee9dc224-ece0-430f-a967-f35905cbdc1a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.595734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.595650    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee9dc224-ece0-430f-a967-f35905cbdc1a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:00:40.960089 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.960053    2559 generic.go:356] "Generic (PLEG): container finished" podID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerID="d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3" exitCode=0
Sep 11 01:00:40.960264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.960100    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerDied","Data":"d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3"}
Sep 11 01:00:40.960264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.960140    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:40.960264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.960182    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee9dc224-ece0-430f-a967-f35905cbdc1a","Type":"ContainerDied","Data":"1567ebacc8aa0f9bd1b5b380b7d0ed704f526f9eb3158f88a9ebca85b62f6bb5"}
Sep 11 01:00:40.960264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.960215    2559 scope.go:117] "RemoveContainer" containerID="6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1"
Sep 11 01:00:40.968007 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.967898    2559 scope.go:117] "RemoveContainer" containerID="13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431"
Sep 11 01:00:40.974792 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.974772    2559 scope.go:117] "RemoveContainer" containerID="2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5"
Sep 11 01:00:40.981089 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.981073    2559 scope.go:117] "RemoveContainer" containerID="d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3"
Sep 11 01:00:40.985307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.985286    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 01:00:40.990146 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.989920    2559 scope.go:117] "RemoveContainer" containerID="d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e"
Sep 11 01:00:40.991866 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.991846    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 01:00:40.997253 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:40.997239    2559 scope.go:117] "RemoveContainer" containerID="1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a"
Sep 11 01:00:41.003931 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.003913    2559 scope.go:117] "RemoveContainer" containerID="ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd"
Sep 11 01:00:41.010501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.010486    2559 scope.go:117] "RemoveContainer" containerID="6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1"
Sep 11 01:00:41.010830 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:00:41.010811    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1\": container with ID starting with 6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1 not found: ID does not exist" containerID="6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1"
Sep 11 01:00:41.010902 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.010839    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1"} err="failed to get container status \"6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1\": rpc error: code = NotFound desc = could not find container \"6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1\": container with ID starting with 6f246b16f68a7796030011df0a72d599986a8f58d24eaddc838b140ea3b53fc1 not found: ID does not exist"
Sep 11 01:00:41.010902 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.010858    2559 scope.go:117] "RemoveContainer" containerID="13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431"
Sep 11 01:00:41.011086 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:00:41.011067    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431\": container with ID starting with 13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431 not found: ID does not exist" containerID="13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431"
Sep 11 01:00:41.011146 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.011098    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431"} err="failed to get container status \"13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431\": rpc error: code = NotFound desc = could not find container \"13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431\": container with ID starting with 13b5014768912976c03da67454f7e98608c65796bddd2926e742588bd93d1431 not found: ID does not exist"
Sep 11 01:00:41.011146 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.011121    2559 scope.go:117] "RemoveContainer" containerID="2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5"
Sep 11 01:00:41.011326 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:00:41.011309    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5\": container with ID starting with 2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5 not found: ID does not exist" containerID="2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5"
Sep 11 01:00:41.011389 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.011334    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5"} err="failed to get container status \"2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5\": rpc error: code = NotFound desc = could not find container \"2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5\": container with ID starting with 2dd54f0348bbe6b87d6890475baeb0c24d246bfd5f48efbc79e14e7905eca5b5 not found: ID does not exist"
Sep 11 01:00:41.011389 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.011355    2559 scope.go:117] "RemoveContainer" containerID="d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3"
Sep 11 01:00:41.011547 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:00:41.011532    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3\": container with ID starting with d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3 not found: ID does not exist" containerID="d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3"
Sep 11 01:00:41.011589 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.011551    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3"} err="failed to get container status \"d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3\": rpc error: code = NotFound desc = could not find container \"d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3\": container with ID starting with d1d74d85f3b30d9735531b0808fb59d6d5b077d59ef5ba362294ad4122c2d9a3 not found: ID does not exist"
Sep 11 01:00:41.011589 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.011563    2559 scope.go:117] "RemoveContainer" containerID="d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e"
Sep 11 01:00:41.011790 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:00:41.011776    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e\": container with ID starting with d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e not found: ID does not exist" containerID="d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e"
Sep 11 01:00:41.011828 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.011792    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e"} err="failed to get container status \"d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e\": rpc error: code = NotFound desc = could not find container \"d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e\": container with ID starting with d8fc2fa9218048a8a7fb94b624e1a4b0ba7dbe9d3c77614dc9670beb40d0297e not found: ID does not exist"
Sep 11 01:00:41.011828 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.011804    2559 scope.go:117] "RemoveContainer" containerID="1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a"
Sep 11 01:00:41.012048 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:00:41.012032    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a\": container with ID starting with 1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a not found: ID does not exist" containerID="1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a"
Sep 11 01:00:41.012108 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.012054    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a"} err="failed to get container status \"1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a\": rpc error: code = NotFound desc = could not find container \"1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a\": container with ID starting with 1b1d402856222d71e449469fc6a04cc4fb1b7b9e0586bee6cd7fcdffe859a41a not found: ID does not exist"
Sep 11 01:00:41.012108 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.012074    2559 scope.go:117] "RemoveContainer" containerID="ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd"
Sep 11 01:00:41.012314 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:00:41.012297    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd\": container with ID starting with ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd not found: ID does not exist" containerID="ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd"
Sep 11 01:00:41.012360 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.012321    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd"} err="failed to get container status \"ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd\": rpc error: code = NotFound desc = could not find container \"ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd\": container with ID starting with ce4e82cabdfc7605d523f183bf8e28fedcba434a6c9707fe72c9e7f6e71aa1bd not found: ID does not exist"
Sep 11 01:00:41.110222 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.110177    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" path="/var/lib/kubelet/pods/ee9dc224-ece0-430f-a967-f35905cbdc1a/volumes"
Sep 11 01:00:41.110775 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.110743    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 01:00:41.111039 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111026    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy-metric"
Sep 11 01:00:41.111094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111041    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy-metric"
Sep 11 01:00:41.111094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111052    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="init-config-reloader"
Sep 11 01:00:41.111094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111057    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="init-config-reloader"
Sep 11 01:00:41.111094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111065    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="config-reloader"
Sep 11 01:00:41.111094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111074    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="config-reloader"
Sep 11 01:00:41.111094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111087    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy-web"
Sep 11 01:00:41.111094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111093    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy-web"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111100    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="alertmanager"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111106    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="alertmanager"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111112    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="prom-label-proxy"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111117    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="prom-label-proxy"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111123    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b8ce73c-a602-4633-9a36-88482852b2cf" containerName="console"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111127    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b8ce73c-a602-4633-9a36-88482852b2cf" containerName="console"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111135    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0481f0ef-71c9-4dae-86ff-13510383e309" containerName="console"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111140    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0481f0ef-71c9-4dae-86ff-13510383e309" containerName="console"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111150    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111158    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111200    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy-metric"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111208    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b8ce73c-a602-4633-9a36-88482852b2cf" containerName="console"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111214    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="alertmanager"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111220    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111229    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="prom-label-proxy"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111237    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0481f0ef-71c9-4dae-86ff-13510383e309" containerName="console"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111242    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="config-reloader"
Sep 11 01:00:41.111293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.111246    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9dc224-ece0-430f-a967-f35905cbdc1a" containerName="kube-rbac-proxy-web"
Sep 11 01:00:41.115347 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.115320    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.120331 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.120309    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 01:00:41.120331 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.120327    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 01:00:41.120860 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.120327    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 01:00:41.120860 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.120377    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 01:00:41.120860 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.120421    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 01:00:41.120860 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.120330    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 01:00:41.120860 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.120423    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 01:00:41.121207 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.121191    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h6mq8\""
Sep 11 01:00:41.121587 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.121564    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 01:00:41.126864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.126845    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 01:00:41.130941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.130919    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 01:00:41.201026 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.200987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201026 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201024    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201222 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201045    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1355c87f-5299-4893-aa7d-e7f07535079b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201222 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201134    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1355c87f-5299-4893-aa7d-e7f07535079b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201222 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201173    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np7c9\" (UniqueName: \"kubernetes.io/projected/1355c87f-5299-4893-aa7d-e7f07535079b-kube-api-access-np7c9\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201222 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201202    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1355c87f-5299-4893-aa7d-e7f07535079b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201371 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201238    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201371 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201256    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1355c87f-5299-4893-aa7d-e7f07535079b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201371 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201294    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201371 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201335    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-config-volume\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201371 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201353    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1355c87f-5299-4893-aa7d-e7f07535079b-config-out\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201647 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201500    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.201647 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.201554    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-web-config\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302399 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302357    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-config-volume\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302399 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1355c87f-5299-4893-aa7d-e7f07535079b-config-out\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302631 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302421    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302631 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302453    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-web-config\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302631 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302479    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302631 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302496    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302631 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302517    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1355c87f-5299-4893-aa7d-e7f07535079b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302631 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302540    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1355c87f-5299-4893-aa7d-e7f07535079b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302955 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302645    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np7c9\" (UniqueName: \"kubernetes.io/projected/1355c87f-5299-4893-aa7d-e7f07535079b-kube-api-access-np7c9\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302955 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302738    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1355c87f-5299-4893-aa7d-e7f07535079b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302955 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302767    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.302955 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1355c87f-5299-4893-aa7d-e7f07535079b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.303141 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.302828    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.303490 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.303466    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1355c87f-5299-4893-aa7d-e7f07535079b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.303914 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.303862    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1355c87f-5299-4893-aa7d-e7f07535079b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.305043 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.305013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1355c87f-5299-4893-aa7d-e7f07535079b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.305698 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.305649    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.305805 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.305723    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-config-volume\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.305805 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.305768    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.306063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.306041    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1355c87f-5299-4893-aa7d-e7f07535079b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.306275 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.306244    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1355c87f-5299-4893-aa7d-e7f07535079b-config-out\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.306363 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.306296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-web-config\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.306363 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.306296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.306725 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.306707    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.307303 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.307286    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1355c87f-5299-4893-aa7d-e7f07535079b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.313904 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.313884    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np7c9\" (UniqueName: \"kubernetes.io/projected/1355c87f-5299-4893-aa7d-e7f07535079b-kube-api-access-np7c9\") pod \"alertmanager-main-0\" (UID: \"1355c87f-5299-4893-aa7d-e7f07535079b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.428310 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.428273    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 01:00:41.710360 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.710326    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 01:00:41.713132 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:00:41.713104    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1355c87f_5299_4893_aa7d_e7f07535079b.slice/crio-2d0cd7a853dfc91d40cd5a0acbab1cc8210fcc30c3a095106fc8fe25c007ceeb WatchSource:0}: Error finding container 2d0cd7a853dfc91d40cd5a0acbab1cc8210fcc30c3a095106fc8fe25c007ceeb: Status 404 returned error can't find the container with id 2d0cd7a853dfc91d40cd5a0acbab1cc8210fcc30c3a095106fc8fe25c007ceeb
Sep 11 01:00:41.965676 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.965577    2559 generic.go:356] "Generic (PLEG): container finished" podID="1355c87f-5299-4893-aa7d-e7f07535079b" containerID="2cc7c57cc8740800096452c2d647c0be79443a7f087ed08561f6fa26e519f787" exitCode=0
Sep 11 01:00:41.965676 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.965648    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1355c87f-5299-4893-aa7d-e7f07535079b","Type":"ContainerDied","Data":"2cc7c57cc8740800096452c2d647c0be79443a7f087ed08561f6fa26e519f787"}
Sep 11 01:00:41.965847 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:41.965727    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1355c87f-5299-4893-aa7d-e7f07535079b","Type":"ContainerStarted","Data":"2d0cd7a853dfc91d40cd5a0acbab1cc8210fcc30c3a095106fc8fe25c007ceeb"}
Sep 11 01:00:42.973721 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:42.973684    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1355c87f-5299-4893-aa7d-e7f07535079b","Type":"ContainerStarted","Data":"166ba44501b3f29eccbd9a5e183145d38b54f1e4cd3c98ae7088c717b68b0757"}
Sep 11 01:00:42.974080 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:42.973764    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1355c87f-5299-4893-aa7d-e7f07535079b","Type":"ContainerStarted","Data":"406ab07ece98ac3f95b0eeae800b292ed2c7cf96318cf6ae171c7bce321d28a6"}
Sep 11 01:00:42.974080 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:42.973810    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1355c87f-5299-4893-aa7d-e7f07535079b","Type":"ContainerStarted","Data":"0b539459305a2b7073f1d664c4059563e5b98ace3d0207e6dd7193a59b0a2bfd"}
Sep 11 01:00:42.974080 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:42.973845    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1355c87f-5299-4893-aa7d-e7f07535079b","Type":"ContainerStarted","Data":"bef7cc22e7e426f230a9186ad92b8e55b2d49ea70b400fe2b39074b3125a68cc"}
Sep 11 01:00:42.974080 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:42.973880    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1355c87f-5299-4893-aa7d-e7f07535079b","Type":"ContainerStarted","Data":"6f1f98dfa3c810397673c736ad1c220a50e98231806ed4d1f6888479014a4ffe"}
Sep 11 01:00:42.974080 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:42.973915    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1355c87f-5299-4893-aa7d-e7f07535079b","Type":"ContainerStarted","Data":"4adcee92f29aa5a7d1beb13d106951734b1efe6579db97aa0ef9dbf47a66220e"}
Sep 11 01:00:43.002036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.001984    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.985214761 podStartE2EDuration="2.001967326s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.752565ms" isStatefulPod=false podCreationTimestamp="2026-09-11 01:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:00:42.999928918 +0000 UTC m=+162.514529100" watchObservedRunningTime="2026-09-11 01:00:43.001967326 +0000 UTC m=+162.516567508"
Sep 11 01:00:43.154639 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.154573    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"]
Sep 11 01:00:43.157262 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.157234    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.160945 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.160284    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 11 01:00:43.160945 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.160574    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 11 01:00:43.160945 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.160807    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 11 01:00:43.161900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.161299    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 11 01:00:43.161900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.161537    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 11 01:00:43.162120 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.162104    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-6jvbj\""
Sep 11 01:00:43.173752 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.173728    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 11 01:00:43.186405 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.186232    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"]
Sep 11 01:00:43.322471 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.322430    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-telemeter-client-tls\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.322808 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.322784    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-secret-telemeter-client\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.322934 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.322914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmx9w\" (UniqueName: \"kubernetes.io/projected/539b9ba0-a678-4b77-bd9c-71bd51985803-kube-api-access-dmx9w\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.323133 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.323073    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b9ba0-a678-4b77-bd9c-71bd51985803-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.323236 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.323133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-federate-client-tls\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.323236 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.323159    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.323236 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.323189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/539b9ba0-a678-4b77-bd9c-71bd51985803-metrics-client-ca\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.323236 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.323234    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b9ba0-a678-4b77-bd9c-71bd51985803-serving-certs-ca-bundle\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.424569 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.424528    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b9ba0-a678-4b77-bd9c-71bd51985803-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.424569 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.424574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-federate-client-tls\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.424837 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.424619    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.424837 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.424660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/539b9ba0-a678-4b77-bd9c-71bd51985803-metrics-client-ca\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.424837 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.424714    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b9ba0-a678-4b77-bd9c-71bd51985803-serving-certs-ca-bundle\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.424995 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.424838    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-telemeter-client-tls\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.424995 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.424894    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-secret-telemeter-client\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.425298 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.425275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmx9w\" (UniqueName: \"kubernetes.io/projected/539b9ba0-a678-4b77-bd9c-71bd51985803-kube-api-access-dmx9w\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.425499 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.425478    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b9ba0-a678-4b77-bd9c-71bd51985803-serving-certs-ca-bundle\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.425943 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.425922    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/539b9ba0-a678-4b77-bd9c-71bd51985803-metrics-client-ca\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.426258 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.426230    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/539b9ba0-a678-4b77-bd9c-71bd51985803-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.427252 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.427229    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-federate-client-tls\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.427394 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.427369    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-telemeter-client-tls\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.427562 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.427541    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.427711 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.427692    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/539b9ba0-a678-4b77-bd9c-71bd51985803-secret-telemeter-client\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.432794 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.432775    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmx9w\" (UniqueName: \"kubernetes.io/projected/539b9ba0-a678-4b77-bd9c-71bd51985803-kube-api-access-dmx9w\") pod \"telemeter-client-7f98bb8f98-qnxp7\" (UID: \"539b9ba0-a678-4b77-bd9c-71bd51985803\") " pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.469830 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.469791    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"
Sep 11 01:00:43.754338 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.754307    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7"]
Sep 11 01:00:43.755159 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:00:43.755131    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod539b9ba0_a678_4b77_bd9c_71bd51985803.slice/crio-fe3b80f5678454cdecafb544f7c8f01a1078015096f289638430ff5db0ca82be WatchSource:0}: Error finding container fe3b80f5678454cdecafb544f7c8f01a1078015096f289638430ff5db0ca82be: Status 404 returned error can't find the container with id fe3b80f5678454cdecafb544f7c8f01a1078015096f289638430ff5db0ca82be
Sep 11 01:00:43.979614 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:43.979581    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7" event={"ID":"539b9ba0-a678-4b77-bd9c-71bd51985803","Type":"ContainerStarted","Data":"fe3b80f5678454cdecafb544f7c8f01a1078015096f289638430ff5db0ca82be"}
Sep 11 01:00:45.992011 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:45.991976    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7" event={"ID":"539b9ba0-a678-4b77-bd9c-71bd51985803","Type":"ContainerStarted","Data":"299e5b52c65435e710b362f052e72233daa5d24c1058e2a4bff5ccbaae155698"}
Sep 11 01:00:45.992428 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:45.992033    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7" event={"ID":"539b9ba0-a678-4b77-bd9c-71bd51985803","Type":"ContainerStarted","Data":"3aac89c7164674ce49aecc68b7ae8007302d95ad3fcf0a2350be1f0f4ce9f8b1"}
Sep 11 01:00:45.992428 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:45.992054    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7" event={"ID":"539b9ba0-a678-4b77-bd9c-71bd51985803","Type":"ContainerStarted","Data":"9d99a9bbd37b4e4d296a321e9aaf33e3070bffbc17c1b5ef267b92428259e9bc"}
Sep 11 01:00:46.015232 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.015177    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f98bb8f98-qnxp7" podStartSLOduration=1.342154634 podStartE2EDuration="3.015160891s" totalImagesPullingTime="1.673006257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:00:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:00:46.014179453 +0000 UTC m=+165.528779639" watchObservedRunningTime="2026-09-11 01:00:46.015160891 +0000 UTC m=+165.529761068"
Sep 11 01:00:46.101847 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.101815    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54f6455c86-2m7qt"]
Sep 11 01:00:46.106935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.105370    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.122141 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.122112    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54f6455c86-2m7qt"]
Sep 11 01:00:46.250711 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.250618    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-config\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.250711 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.250670    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-service-ca\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.250711 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.250694    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-serving-cert\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.250931 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.250715    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-oauth-config\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.250931 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.250809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-oauth-serving-cert\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.250931 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.250882    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74pqd\" (UniqueName: \"kubernetes.io/projected/a2285ced-16b7-434c-a1e6-9e3a33621a4b-kube-api-access-74pqd\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.250931 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.250898    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-trusted-ca-bundle\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.351472 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.351433    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74pqd\" (UniqueName: \"kubernetes.io/projected/a2285ced-16b7-434c-a1e6-9e3a33621a4b-kube-api-access-74pqd\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.351472 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.351471    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-trusted-ca-bundle\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.351770 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.351497    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-config\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.351770 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.351512    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-service-ca\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.351770 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.351532    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-serving-cert\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.351770 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.351673    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-oauth-config\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.351770 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.351748    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-oauth-serving-cert\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.352322 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.352291    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-service-ca\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.352442 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.352342    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-config\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.352492 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.352458    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-oauth-serving-cert\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.352492 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.352480    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-trusted-ca-bundle\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.354137 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.354114    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-oauth-config\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.354462 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.354440    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-serving-cert\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.364855 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.364834    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74pqd\" (UniqueName: \"kubernetes.io/projected/a2285ced-16b7-434c-a1e6-9e3a33621a4b-kube-api-access-74pqd\") pod \"console-54f6455c86-2m7qt\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") " pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.415983 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.415942    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:46.707278 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.707240    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54f6455c86-2m7qt"]
Sep 11 01:00:46.710780 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:00:46.710751    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2285ced_16b7_434c_a1e6_9e3a33621a4b.slice/crio-68afec8ccfd43147879830ab898f0c5b31604eb5b97fd7a22dc019b5ed751dee WatchSource:0}: Error finding container 68afec8ccfd43147879830ab898f0c5b31604eb5b97fd7a22dc019b5ed751dee: Status 404 returned error can't find the container with id 68afec8ccfd43147879830ab898f0c5b31604eb5b97fd7a22dc019b5ed751dee
Sep 11 01:00:46.998085 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.997998    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f6455c86-2m7qt" event={"ID":"a2285ced-16b7-434c-a1e6-9e3a33621a4b","Type":"ContainerStarted","Data":"121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b"}
Sep 11 01:00:46.998085 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:46.998053    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f6455c86-2m7qt" event={"ID":"a2285ced-16b7-434c-a1e6-9e3a33621a4b","Type":"ContainerStarted","Data":"68afec8ccfd43147879830ab898f0c5b31604eb5b97fd7a22dc019b5ed751dee"}
Sep 11 01:00:47.028156 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:47.028105    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54f6455c86-2m7qt" podStartSLOduration=1.028088157 podStartE2EDuration="1.028088157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:00:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:00:47.027415216 +0000 UTC m=+166.542015399" watchObservedRunningTime="2026-09-11 01:00:47.028088157 +0000 UTC m=+166.542688351"
Sep 11 01:00:56.416568 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:56.416472    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:56.416568 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:56.416549    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:56.421179 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:56.421157    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:57.042116 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:57.042087    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:00:57.084866 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:00:57.084827    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fc877f5b-sv4xg"]
Sep 11 01:01:22.107299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.107229    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fc877f5b-sv4xg" podUID="cef27599-ba2c-48a7-a7ef-2ef35ee8254c" containerName="console" containerID="cri-o://5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60" gracePeriod=15
Sep 11 01:01:22.344900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.344877    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 01:01:22.454832 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.454742    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-serving-cert\") pod \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") "
Sep 11 01:01:22.454832 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.454809    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-oauth-config\") pod \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") "
Sep 11 01:01:22.455063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.454847    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-service-ca\") pod \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") "
Sep 11 01:01:22.455063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.454893    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-oauth-serving-cert\") pod \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") "
Sep 11 01:01:22.455063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.454925    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj5wn\" (UniqueName: \"kubernetes.io/projected/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-kube-api-access-hj5wn\") pod \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") "
Sep 11 01:01:22.455063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.454956    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-trusted-ca-bundle\") pod \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") "
Sep 11 01:01:22.455063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.454981    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-config\") pod \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\" (UID: \"cef27599-ba2c-48a7-a7ef-2ef35ee8254c\") "
Sep 11 01:01:22.455581 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.455294    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-service-ca" (OuterVolumeSpecName: "service-ca") pod "cef27599-ba2c-48a7-a7ef-2ef35ee8254c" (UID: "cef27599-ba2c-48a7-a7ef-2ef35ee8254c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:01:22.455581 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.455386    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cef27599-ba2c-48a7-a7ef-2ef35ee8254c" (UID: "cef27599-ba2c-48a7-a7ef-2ef35ee8254c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:01:22.455581 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.455512    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-config" (OuterVolumeSpecName: "console-config") pod "cef27599-ba2c-48a7-a7ef-2ef35ee8254c" (UID: "cef27599-ba2c-48a7-a7ef-2ef35ee8254c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:01:22.455581 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.455542    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cef27599-ba2c-48a7-a7ef-2ef35ee8254c" (UID: "cef27599-ba2c-48a7-a7ef-2ef35ee8254c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:01:22.457072 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.457049    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cef27599-ba2c-48a7-a7ef-2ef35ee8254c" (UID: "cef27599-ba2c-48a7-a7ef-2ef35ee8254c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:01:22.457072 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.457062    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cef27599-ba2c-48a7-a7ef-2ef35ee8254c" (UID: "cef27599-ba2c-48a7-a7ef-2ef35ee8254c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:01:22.457234 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.457067    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-kube-api-access-hj5wn" (OuterVolumeSpecName: "kube-api-access-hj5wn") pod "cef27599-ba2c-48a7-a7ef-2ef35ee8254c" (UID: "cef27599-ba2c-48a7-a7ef-2ef35ee8254c"). InnerVolumeSpecName "kube-api-access-hj5wn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:01:22.556194 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.556152    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-serving-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:01:22.556194 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.556185    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-oauth-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:01:22.556194 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.556197    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:01:22.556437 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.556210    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-oauth-serving-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:01:22.556437 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.556221    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj5wn\" (UniqueName: \"kubernetes.io/projected/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-kube-api-access-hj5wn\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:01:22.556437 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.556232    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-trusted-ca-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:01:22.556437 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:22.556243    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef27599-ba2c-48a7-a7ef-2ef35ee8254c-console-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:01:23.139983 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:23.139952    2559 generic.go:356] "Generic (PLEG): container finished" podID="cef27599-ba2c-48a7-a7ef-2ef35ee8254c" containerID="5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60" exitCode=0
Sep 11 01:01:23.140362 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:23.139990    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fc877f5b-sv4xg"
Sep 11 01:01:23.140362 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:23.140037    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc877f5b-sv4xg" event={"ID":"cef27599-ba2c-48a7-a7ef-2ef35ee8254c","Type":"ContainerDied","Data":"5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60"}
Sep 11 01:01:23.140362 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:23.140083    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fc877f5b-sv4xg" event={"ID":"cef27599-ba2c-48a7-a7ef-2ef35ee8254c","Type":"ContainerDied","Data":"ff5cc0cb8f699a4fae15c59665ef011b046f7fef2be4f94790e9d1636926d721"}
Sep 11 01:01:23.140362 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:23.140102    2559 scope.go:117] "RemoveContainer" containerID="5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60"
Sep 11 01:01:23.148056 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:23.148040    2559 scope.go:117] "RemoveContainer" containerID="5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60"
Sep 11 01:01:23.148285 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:01:23.148268    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60\": container with ID starting with 5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60 not found: ID does not exist" containerID="5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60"
Sep 11 01:01:23.148328 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:23.148293    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60"} err="failed to get container status \"5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60\": rpc error: code = NotFound desc = could not find container \"5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60\": container with ID starting with 5647e29f66203ddc6a8bf5c971554a44bfb82cbffac3e52e4bf19a205587fa60 not found: ID does not exist"
Sep 11 01:01:23.157929 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:23.157907    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fc877f5b-sv4xg"]
Sep 11 01:01:23.161884 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:23.161862    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fc877f5b-sv4xg"]
Sep 11 01:01:25.108948 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:25.108910    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cef27599-ba2c-48a7-a7ef-2ef35ee8254c" path="/var/lib/kubelet/pods/cef27599-ba2c-48a7-a7ef-2ef35ee8254c/volumes"
Sep 11 01:01:56.881170 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.881132    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-549c87c684-d425s"]
Sep 11 01:01:56.881700 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.881483    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef27599-ba2c-48a7-a7ef-2ef35ee8254c" containerName="console"
Sep 11 01:01:56.881700 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.881496    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef27599-ba2c-48a7-a7ef-2ef35ee8254c" containerName="console"
Sep 11 01:01:56.881700 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.881538    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cef27599-ba2c-48a7-a7ef-2ef35ee8254c" containerName="console"
Sep 11 01:01:56.883792 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.883777    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:56.897585 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.897554    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-549c87c684-d425s"]
Sep 11 01:01:56.938014 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.937985    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-service-ca\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:56.938169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.938047    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqvx6\" (UniqueName: \"kubernetes.io/projected/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-kube-api-access-sqvx6\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:56.938169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.938073    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-config\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:56.938169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.938094    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-serving-cert\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:56.938169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.938109    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-oauth-config\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:56.938169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.938152    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-oauth-serving-cert\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:56.938369 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:56.938208    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-trusted-ca-bundle\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.038895 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.038851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-service-ca\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.039082 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.038924    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqvx6\" (UniqueName: \"kubernetes.io/projected/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-kube-api-access-sqvx6\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.039082 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.038948    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-config\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.039082 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.038974    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-serving-cert\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.039082 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.038999    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-oauth-config\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.039082 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.039055    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-oauth-serving-cert\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.039344 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.039085    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-trusted-ca-bundle\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.039687 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.039663    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-service-ca\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.040173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.040148    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-config\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.040245 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.040193    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-oauth-serving-cert\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.040453 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.040431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-trusted-ca-bundle\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.041582 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.041561    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-serving-cert\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.041679 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.041616    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-oauth-config\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.047192 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.047157    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqvx6\" (UniqueName: \"kubernetes.io/projected/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-kube-api-access-sqvx6\") pod \"console-549c87c684-d425s\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") " pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.193079 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.192988    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:01:57.462169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:57.462122    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-549c87c684-d425s"]
Sep 11 01:01:57.464655 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:01:57.464622    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2111085_c0b7_4a9c_9cd1_44ef07adaf9c.slice/crio-316ac4c70f67bf429530ad0956f30a33e1817976ab3807b8e04945aa238c53d9 WatchSource:0}: Error finding container 316ac4c70f67bf429530ad0956f30a33e1817976ab3807b8e04945aa238c53d9: Status 404 returned error can't find the container with id 316ac4c70f67bf429530ad0956f30a33e1817976ab3807b8e04945aa238c53d9
Sep 11 01:01:58.277747 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:58.277712    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549c87c684-d425s" event={"ID":"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c","Type":"ContainerStarted","Data":"f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9"}
Sep 11 01:01:58.278121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:58.277761    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549c87c684-d425s" event={"ID":"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c","Type":"ContainerStarted","Data":"316ac4c70f67bf429530ad0956f30a33e1817976ab3807b8e04945aa238c53d9"}
Sep 11 01:01:58.297772 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:01:58.297722    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-549c87c684-d425s" podStartSLOduration=2.297707084 podStartE2EDuration="2.297707084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:01:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:01:58.296537392 +0000 UTC m=+237.811137591" watchObservedRunningTime="2026-09-11 01:01:58.297707084 +0000 UTC m=+237.812307269"
Sep 11 01:02:07.193279 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:07.193231    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:02:07.193708 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:07.193380    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:02:07.198532 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:07.198505    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:02:07.313943 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:07.313916    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:02:07.362946 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:07.362913    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54f6455c86-2m7qt"]
Sep 11 01:02:32.382683 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.382578    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54f6455c86-2m7qt" podUID="a2285ced-16b7-434c-a1e6-9e3a33621a4b" containerName="console" containerID="cri-o://121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b" gracePeriod=15
Sep 11 01:02:32.626143 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.626118    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:02:32.727864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.727782    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-oauth-serving-cert\") pod \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") "
Sep 11 01:02:32.727864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.727820    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-config\") pod \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") "
Sep 11 01:02:32.727864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.727845    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-serving-cert\") pod \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") "
Sep 11 01:02:32.727864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.727863    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-trusted-ca-bundle\") pod \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") "
Sep 11 01:02:32.728209 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.728064    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-oauth-config\") pod \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") "
Sep 11 01:02:32.728209 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.728107    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74pqd\" (UniqueName: \"kubernetes.io/projected/a2285ced-16b7-434c-a1e6-9e3a33621a4b-kube-api-access-74pqd\") pod \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") "
Sep 11 01:02:32.728209 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.728143    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-service-ca\") pod \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\" (UID: \"a2285ced-16b7-434c-a1e6-9e3a33621a4b\") "
Sep 11 01:02:32.728349 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.728257    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-config" (OuterVolumeSpecName: "console-config") pod "a2285ced-16b7-434c-a1e6-9e3a33621a4b" (UID: "a2285ced-16b7-434c-a1e6-9e3a33621a4b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:02:32.728349 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.728280    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a2285ced-16b7-434c-a1e6-9e3a33621a4b" (UID: "a2285ced-16b7-434c-a1e6-9e3a33621a4b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:02:32.728349 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.728310    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a2285ced-16b7-434c-a1e6-9e3a33621a4b" (UID: "a2285ced-16b7-434c-a1e6-9e3a33621a4b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:02:32.728624 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.728584    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-oauth-serving-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:02:32.728728 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.728595    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-service-ca" (OuterVolumeSpecName: "service-ca") pod "a2285ced-16b7-434c-a1e6-9e3a33621a4b" (UID: "a2285ced-16b7-434c-a1e6-9e3a33621a4b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:02:32.728728 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.728629    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:02:32.728728 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.728669    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-trusted-ca-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:02:32.730014 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.729987    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a2285ced-16b7-434c-a1e6-9e3a33621a4b" (UID: "a2285ced-16b7-434c-a1e6-9e3a33621a4b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:02:32.730177 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.730155    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a2285ced-16b7-434c-a1e6-9e3a33621a4b" (UID: "a2285ced-16b7-434c-a1e6-9e3a33621a4b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:02:32.730222 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.730165    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2285ced-16b7-434c-a1e6-9e3a33621a4b-kube-api-access-74pqd" (OuterVolumeSpecName: "kube-api-access-74pqd") pod "a2285ced-16b7-434c-a1e6-9e3a33621a4b" (UID: "a2285ced-16b7-434c-a1e6-9e3a33621a4b"). InnerVolumeSpecName "kube-api-access-74pqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:02:32.829663 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.829627    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2285ced-16b7-434c-a1e6-9e3a33621a4b-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:02:32.829663 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.829656    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-serving-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:02:32.829663 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.829666    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2285ced-16b7-434c-a1e6-9e3a33621a4b-console-oauth-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:02:32.829878 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:32.829675    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74pqd\" (UniqueName: \"kubernetes.io/projected/a2285ced-16b7-434c-a1e6-9e3a33621a4b-kube-api-access-74pqd\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:02:33.414652 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:33.414554    2559 generic.go:356] "Generic (PLEG): container finished" podID="a2285ced-16b7-434c-a1e6-9e3a33621a4b" containerID="121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b" exitCode=0
Sep 11 01:02:33.414652 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:33.414600    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f6455c86-2m7qt"
Sep 11 01:02:33.414652 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:33.414642    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f6455c86-2m7qt" event={"ID":"a2285ced-16b7-434c-a1e6-9e3a33621a4b","Type":"ContainerDied","Data":"121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b"}
Sep 11 01:02:33.415153 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:33.414696    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f6455c86-2m7qt" event={"ID":"a2285ced-16b7-434c-a1e6-9e3a33621a4b","Type":"ContainerDied","Data":"68afec8ccfd43147879830ab898f0c5b31604eb5b97fd7a22dc019b5ed751dee"}
Sep 11 01:02:33.415153 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:33.414725    2559 scope.go:117] "RemoveContainer" containerID="121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b"
Sep 11 01:02:33.422800 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:33.422784    2559 scope.go:117] "RemoveContainer" containerID="121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b"
Sep 11 01:02:33.423079 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:02:33.423056    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b\": container with ID starting with 121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b not found: ID does not exist" containerID="121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b"
Sep 11 01:02:33.423137 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:33.423090    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b"} err="failed to get container status \"121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b\": rpc error: code = NotFound desc = could not find container \"121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b\": container with ID starting with 121f19627c5ae7efa1004c58e4d5598823ea3fd8eb85269abca0e470d84c9b2b not found: ID does not exist"
Sep 11 01:02:33.431054 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:33.431028    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54f6455c86-2m7qt"]
Sep 11 01:02:33.437281 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:33.437258    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54f6455c86-2m7qt"]
Sep 11 01:02:35.109047 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:02:35.109013    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2285ced-16b7-434c-a1e6-9e3a33621a4b" path="/var/lib/kubelet/pods/a2285ced-16b7-434c-a1e6-9e3a33621a4b/volumes"
Sep 11 01:03:00.901120 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:00.900999    2559 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 01:03:01.249899 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.249696    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"]
Sep 11 01:03:01.263448 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.250015    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2285ced-16b7-434c-a1e6-9e3a33621a4b" containerName="console"
Sep 11 01:03:01.263448 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.250027    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2285ced-16b7-434c-a1e6-9e3a33621a4b" containerName="console"
Sep 11 01:03:01.263448 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.250104    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2285ced-16b7-434c-a1e6-9e3a33621a4b" containerName="console"
Sep 11 01:03:01.263448 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.255106    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.263448 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.258008    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w87xq\""
Sep 11 01:03:01.263448 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.258033    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 01:03:01.263448 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.259039    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 01:03:01.263448 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.263161    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"]
Sep 11 01:03:01.335657 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.335624    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvspb\" (UniqueName: \"kubernetes.io/projected/36eb879a-bb71-4abf-8034-d980ee815393-kube-api-access-vvspb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.335811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.335710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.335811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.335749    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.436477 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.436425    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvspb\" (UniqueName: \"kubernetes.io/projected/36eb879a-bb71-4abf-8034-d980ee815393-kube-api-access-vvspb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.436694 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.436502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.436694 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.436525    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.436922 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.436905    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.436959 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.436948    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.444784 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.444752    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvspb\" (UniqueName: \"kubernetes.io/projected/36eb879a-bb71-4abf-8034-d980ee815393-kube-api-access-vvspb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.565175 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.565140    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:01.833745 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:01.833717    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"]
Sep 11 01:03:01.836262 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:03:01.836235    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36eb879a_bb71_4abf_8034_d980ee815393.slice/crio-c3436c9ed82cb372995d48d5275ababc6ddafbcfa87a406cbe6736e58cd5cd1d WatchSource:0}: Error finding container c3436c9ed82cb372995d48d5275ababc6ddafbcfa87a406cbe6736e58cd5cd1d: Status 404 returned error can't find the container with id c3436c9ed82cb372995d48d5275ababc6ddafbcfa87a406cbe6736e58cd5cd1d
Sep 11 01:03:02.524436 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:02.524398    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v" event={"ID":"36eb879a-bb71-4abf-8034-d980ee815393","Type":"ContainerStarted","Data":"c3436c9ed82cb372995d48d5275ababc6ddafbcfa87a406cbe6736e58cd5cd1d"}
Sep 11 01:03:08.550645 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:08.550590    2559 generic.go:356] "Generic (PLEG): container finished" podID="36eb879a-bb71-4abf-8034-d980ee815393" containerID="5c6c1f8c6c55cb429db3c050ecaf4de8939dbc24f12041f800b9fe899091b8c1" exitCode=0
Sep 11 01:03:08.550645 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:08.550634    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v" event={"ID":"36eb879a-bb71-4abf-8034-d980ee815393","Type":"ContainerDied","Data":"5c6c1f8c6c55cb429db3c050ecaf4de8939dbc24f12041f800b9fe899091b8c1"}
Sep 11 01:03:08.551685 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:08.551665    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 01:03:11.566890 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:11.566853    2559 generic.go:356] "Generic (PLEG): container finished" podID="36eb879a-bb71-4abf-8034-d980ee815393" containerID="22b773af8521d4c3d7621675891a9f690091ee7eaaf876289bf07085a17abe94" exitCode=0
Sep 11 01:03:11.567262 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:11.566948    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v" event={"ID":"36eb879a-bb71-4abf-8034-d980ee815393","Type":"ContainerDied","Data":"22b773af8521d4c3d7621675891a9f690091ee7eaaf876289bf07085a17abe94"}
Sep 11 01:03:20.605367 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:20.605333    2559 generic.go:356] "Generic (PLEG): container finished" podID="36eb879a-bb71-4abf-8034-d980ee815393" containerID="e75f5c2c577c16d21021620d44139f65539ddf6431a9adb61eecbe0017ca0e02" exitCode=0
Sep 11 01:03:20.605855 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:20.605395    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v" event={"ID":"36eb879a-bb71-4abf-8034-d980ee815393","Type":"ContainerDied","Data":"e75f5c2c577c16d21021620d44139f65539ddf6431a9adb61eecbe0017ca0e02"}
Sep 11 01:03:21.746357 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:21.746333    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:21.820287 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:21.820262    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvspb\" (UniqueName: \"kubernetes.io/projected/36eb879a-bb71-4abf-8034-d980ee815393-kube-api-access-vvspb\") pod \"36eb879a-bb71-4abf-8034-d980ee815393\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") "
Sep 11 01:03:21.820468 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:21.820311    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-util\") pod \"36eb879a-bb71-4abf-8034-d980ee815393\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") "
Sep 11 01:03:21.820468 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:21.820355    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-bundle\") pod \"36eb879a-bb71-4abf-8034-d980ee815393\" (UID: \"36eb879a-bb71-4abf-8034-d980ee815393\") "
Sep 11 01:03:21.821081 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:21.821047    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-bundle" (OuterVolumeSpecName: "bundle") pod "36eb879a-bb71-4abf-8034-d980ee815393" (UID: "36eb879a-bb71-4abf-8034-d980ee815393"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:03:21.822452 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:21.822420    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36eb879a-bb71-4abf-8034-d980ee815393-kube-api-access-vvspb" (OuterVolumeSpecName: "kube-api-access-vvspb") pod "36eb879a-bb71-4abf-8034-d980ee815393" (UID: "36eb879a-bb71-4abf-8034-d980ee815393"). InnerVolumeSpecName "kube-api-access-vvspb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:03:21.824381 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:21.824356    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-util" (OuterVolumeSpecName: "util") pod "36eb879a-bb71-4abf-8034-d980ee815393" (UID: "36eb879a-bb71-4abf-8034-d980ee815393"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:03:21.921187 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:21.921113    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvspb\" (UniqueName: \"kubernetes.io/projected/36eb879a-bb71-4abf-8034-d980ee815393-kube-api-access-vvspb\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:03:21.921187 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:21.921141    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-util\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:03:21.921187 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:21.921152    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36eb879a-bb71-4abf-8034-d980ee815393-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:03:22.616024 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:22.615996    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v"
Sep 11 01:03:22.616193 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:22.615995    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409vl8v" event={"ID":"36eb879a-bb71-4abf-8034-d980ee815393","Type":"ContainerDied","Data":"c3436c9ed82cb372995d48d5275ababc6ddafbcfa87a406cbe6736e58cd5cd1d"}
Sep 11 01:03:22.616193 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:22.616115    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3436c9ed82cb372995d48d5275ababc6ddafbcfa87a406cbe6736e58cd5cd1d"
Sep 11 01:03:29.890256 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.890220    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"]
Sep 11 01:03:29.890870 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.890546    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36eb879a-bb71-4abf-8034-d980ee815393" containerName="pull"
Sep 11 01:03:29.890870 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.890559    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="36eb879a-bb71-4abf-8034-d980ee815393" containerName="pull"
Sep 11 01:03:29.890870 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.890567    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36eb879a-bb71-4abf-8034-d980ee815393" containerName="extract"
Sep 11 01:03:29.890870 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.890572    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="36eb879a-bb71-4abf-8034-d980ee815393" containerName="extract"
Sep 11 01:03:29.890870 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.890581    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36eb879a-bb71-4abf-8034-d980ee815393" containerName="util"
Sep 11 01:03:29.890870 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.890588    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="36eb879a-bb71-4abf-8034-d980ee815393" containerName="util"
Sep 11 01:03:29.890870 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.890667    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="36eb879a-bb71-4abf-8034-d980ee815393" containerName="extract"
Sep 11 01:03:29.892637 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.892620    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"
Sep 11 01:03:29.896948 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.896925    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 11 01:03:29.897069 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.896979    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 11 01:03:29.897375 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.897353    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-g4mpq\""
Sep 11 01:03:29.912634 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.912589    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"]
Sep 11 01:03:29.985382 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.985296    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457234b9-b140-46f4-b8fc-edb4bbcef11c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fdv9n\" (UID: \"457234b9-b140-46f4-b8fc-edb4bbcef11c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"
Sep 11 01:03:29.985382 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:29.985342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjsvl\" (UniqueName: \"kubernetes.io/projected/457234b9-b140-46f4-b8fc-edb4bbcef11c-kube-api-access-tjsvl\") pod \"cert-manager-operator-controller-manager-55c444bf86-fdv9n\" (UID: \"457234b9-b140-46f4-b8fc-edb4bbcef11c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"
Sep 11 01:03:30.086695 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:30.086644    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457234b9-b140-46f4-b8fc-edb4bbcef11c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fdv9n\" (UID: \"457234b9-b140-46f4-b8fc-edb4bbcef11c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"
Sep 11 01:03:30.086888 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:30.086702    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjsvl\" (UniqueName: \"kubernetes.io/projected/457234b9-b140-46f4-b8fc-edb4bbcef11c-kube-api-access-tjsvl\") pod \"cert-manager-operator-controller-manager-55c444bf86-fdv9n\" (UID: \"457234b9-b140-46f4-b8fc-edb4bbcef11c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"
Sep 11 01:03:30.087190 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:30.087165    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/457234b9-b140-46f4-b8fc-edb4bbcef11c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fdv9n\" (UID: \"457234b9-b140-46f4-b8fc-edb4bbcef11c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"
Sep 11 01:03:30.104465 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:30.104435    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjsvl\" (UniqueName: \"kubernetes.io/projected/457234b9-b140-46f4-b8fc-edb4bbcef11c-kube-api-access-tjsvl\") pod \"cert-manager-operator-controller-manager-55c444bf86-fdv9n\" (UID: \"457234b9-b140-46f4-b8fc-edb4bbcef11c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"
Sep 11 01:03:30.201041 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:30.200958    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"
Sep 11 01:03:30.497307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:30.497228    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n"]
Sep 11 01:03:30.514337 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:03:30.514306    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457234b9_b140_46f4_b8fc_edb4bbcef11c.slice/crio-47e9814c160fd6affed87e72e2d2e352f74f72f4dd7c7a4b928c3e190129b586 WatchSource:0}: Error finding container 47e9814c160fd6affed87e72e2d2e352f74f72f4dd7c7a4b928c3e190129b586: Status 404 returned error can't find the container with id 47e9814c160fd6affed87e72e2d2e352f74f72f4dd7c7a4b928c3e190129b586
Sep 11 01:03:30.649415 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:30.649379    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n" event={"ID":"457234b9-b140-46f4-b8fc-edb4bbcef11c","Type":"ContainerStarted","Data":"47e9814c160fd6affed87e72e2d2e352f74f72f4dd7c7a4b928c3e190129b586"}
Sep 11 01:03:32.662280 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:32.662235    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n" event={"ID":"457234b9-b140-46f4-b8fc-edb4bbcef11c","Type":"ContainerStarted","Data":"298b12979062210b2988d57fd859d3a690c3880292c5d91f978803a7d7e7d8d7"}
Sep 11 01:03:32.690128 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:32.690074    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fdv9n" podStartSLOduration=2.250318541 podStartE2EDuration="3.69005401s" totalImagesPullingTime="1.439735469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:03:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:03:32.688141781 +0000 UTC m=+332.202741966" watchObservedRunningTime="2026-09-11 01:03:32.69005401 +0000 UTC m=+332.204654193"
Sep 11 01:03:33.310077 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.310039    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"]
Sep 11 01:03:33.312853 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.312830    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.315973 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.315948    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w87xq\""
Sep 11 01:03:33.316993 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.316974    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 01:03:33.317065 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.317004    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 01:03:33.329210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.329170    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"]
Sep 11 01:03:33.419402 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.419367    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.419633 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.419439    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.419633 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.419476    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dfzl\" (UniqueName: \"kubernetes.io/projected/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-kube-api-access-5dfzl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.520443 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.520406    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.520443 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.520445    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dfzl\" (UniqueName: \"kubernetes.io/projected/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-kube-api-access-5dfzl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.520676 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.520488    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.520823 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.520797    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.520863 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.520835    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.536822 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.536781    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dfzl\" (UniqueName: \"kubernetes.io/projected/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-kube-api-access-5dfzl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.622403 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.622328    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:33.901674 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:33.901651    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"]
Sep 11 01:03:33.904003 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:03:33.903965    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd635b7_4f9f_4c62_9914_54c81aa58ac8.slice/crio-0f903b5badb3d175140635265c6dfd3f22633eaf651ea6ca9e6eac1a8638a5a3 WatchSource:0}: Error finding container 0f903b5badb3d175140635265c6dfd3f22633eaf651ea6ca9e6eac1a8638a5a3: Status 404 returned error can't find the container with id 0f903b5badb3d175140635265c6dfd3f22633eaf651ea6ca9e6eac1a8638a5a3
Sep 11 01:03:34.672566 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:34.672536    2559 generic.go:356] "Generic (PLEG): container finished" podID="2dd635b7-4f9f-4c62-9914-54c81aa58ac8" containerID="97d5ec333287a57c332eae45e74126822e74ce1efb88ff9310353eaea3c4d289" exitCode=0
Sep 11 01:03:34.672738 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:34.672641    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz" event={"ID":"2dd635b7-4f9f-4c62-9914-54c81aa58ac8","Type":"ContainerDied","Data":"97d5ec333287a57c332eae45e74126822e74ce1efb88ff9310353eaea3c4d289"}
Sep 11 01:03:34.672738 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:34.672692    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz" event={"ID":"2dd635b7-4f9f-4c62-9914-54c81aa58ac8","Type":"ContainerStarted","Data":"0f903b5badb3d175140635265c6dfd3f22633eaf651ea6ca9e6eac1a8638a5a3"}
Sep 11 01:03:36.243202 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.243165    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"]
Sep 11 01:03:36.250750 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.250724    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"
Sep 11 01:03:36.253919 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.253895    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 01:03:36.254122 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.254068    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 01:03:36.255370 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.255173    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-bzfwm\""
Sep 11 01:03:36.258619 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.258569    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"]
Sep 11 01:03:36.344010 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.343957    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2pst\" (UniqueName: \"kubernetes.io/projected/44ef9628-d7d1-4e09-a3b5-9b9e7d9df427-kube-api-access-m2pst\") pod \"cert-manager-cainjector-559cdd8c87-k7ljn\" (UID: \"44ef9628-d7d1-4e09-a3b5-9b9e7d9df427\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"
Sep 11 01:03:36.344190 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.344075    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44ef9628-d7d1-4e09-a3b5-9b9e7d9df427-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-k7ljn\" (UID: \"44ef9628-d7d1-4e09-a3b5-9b9e7d9df427\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"
Sep 11 01:03:36.445027 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.444992    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44ef9628-d7d1-4e09-a3b5-9b9e7d9df427-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-k7ljn\" (UID: \"44ef9628-d7d1-4e09-a3b5-9b9e7d9df427\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"
Sep 11 01:03:36.445191 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.445064    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2pst\" (UniqueName: \"kubernetes.io/projected/44ef9628-d7d1-4e09-a3b5-9b9e7d9df427-kube-api-access-m2pst\") pod \"cert-manager-cainjector-559cdd8c87-k7ljn\" (UID: \"44ef9628-d7d1-4e09-a3b5-9b9e7d9df427\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"
Sep 11 01:03:36.453273 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.453243    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44ef9628-d7d1-4e09-a3b5-9b9e7d9df427-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-k7ljn\" (UID: \"44ef9628-d7d1-4e09-a3b5-9b9e7d9df427\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"
Sep 11 01:03:36.453515 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.453488    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2pst\" (UniqueName: \"kubernetes.io/projected/44ef9628-d7d1-4e09-a3b5-9b9e7d9df427-kube-api-access-m2pst\") pod \"cert-manager-cainjector-559cdd8c87-k7ljn\" (UID: \"44ef9628-d7d1-4e09-a3b5-9b9e7d9df427\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"
Sep 11 01:03:36.576197 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.576172    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"
Sep 11 01:03:36.684639 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.684581    2559 generic.go:356] "Generic (PLEG): container finished" podID="2dd635b7-4f9f-4c62-9914-54c81aa58ac8" containerID="f892df0f17386deba9e99ad94ad1b31abd900dbf442fd0af9130b61623386d9f" exitCode=0
Sep 11 01:03:36.684785 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.684637    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz" event={"ID":"2dd635b7-4f9f-4c62-9914-54c81aa58ac8","Type":"ContainerDied","Data":"f892df0f17386deba9e99ad94ad1b31abd900dbf442fd0af9130b61623386d9f"}
Sep 11 01:03:36.855071 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:36.855038    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn"]
Sep 11 01:03:36.870386 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:03:36.870357    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44ef9628_d7d1_4e09_a3b5_9b9e7d9df427.slice/crio-b11f4486cbf6989a9b6f070f418525bcb907490137302f84886a47fa6d966227 WatchSource:0}: Error finding container b11f4486cbf6989a9b6f070f418525bcb907490137302f84886a47fa6d966227: Status 404 returned error can't find the container with id b11f4486cbf6989a9b6f070f418525bcb907490137302f84886a47fa6d966227
Sep 11 01:03:37.692854 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:37.692818    2559 generic.go:356] "Generic (PLEG): container finished" podID="2dd635b7-4f9f-4c62-9914-54c81aa58ac8" containerID="76e0c1586c2b8270cf97409291427cfe08a225cd5b5f5ab06f225edc2ad57f14" exitCode=0
Sep 11 01:03:37.693271 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:37.692920    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz" event={"ID":"2dd635b7-4f9f-4c62-9914-54c81aa58ac8","Type":"ContainerDied","Data":"76e0c1586c2b8270cf97409291427cfe08a225cd5b5f5ab06f225edc2ad57f14"}
Sep 11 01:03:37.694259 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:37.694230    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn" event={"ID":"44ef9628-d7d1-4e09-a3b5-9b9e7d9df427","Type":"ContainerStarted","Data":"b11f4486cbf6989a9b6f070f418525bcb907490137302f84886a47fa6d966227"}
Sep 11 01:03:39.471884 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.471860    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:39.572103 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.572075    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-util\") pod \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") "
Sep 11 01:03:39.572255 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.572190    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dfzl\" (UniqueName: \"kubernetes.io/projected/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-kube-api-access-5dfzl\") pod \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") "
Sep 11 01:03:39.572255 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.572217    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-bundle\") pod \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\" (UID: \"2dd635b7-4f9f-4c62-9914-54c81aa58ac8\") "
Sep 11 01:03:39.572754 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.572723    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-bundle" (OuterVolumeSpecName: "bundle") pod "2dd635b7-4f9f-4c62-9914-54c81aa58ac8" (UID: "2dd635b7-4f9f-4c62-9914-54c81aa58ac8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:03:39.574284 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.574257    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-kube-api-access-5dfzl" (OuterVolumeSpecName: "kube-api-access-5dfzl") pod "2dd635b7-4f9f-4c62-9914-54c81aa58ac8" (UID: "2dd635b7-4f9f-4c62-9914-54c81aa58ac8"). InnerVolumeSpecName "kube-api-access-5dfzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:03:39.576473 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.576435    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-util" (OuterVolumeSpecName: "util") pod "2dd635b7-4f9f-4c62-9914-54c81aa58ac8" (UID: "2dd635b7-4f9f-4c62-9914-54c81aa58ac8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:03:39.673413 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.673373    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dfzl\" (UniqueName: \"kubernetes.io/projected/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-kube-api-access-5dfzl\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:03:39.673413 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.673406    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:03:39.673413 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.673415    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2dd635b7-4f9f-4c62-9914-54c81aa58ac8-util\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:03:39.705623 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.705519    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz"
Sep 11 01:03:39.705623 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.705490    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fn8rlz" event={"ID":"2dd635b7-4f9f-4c62-9914-54c81aa58ac8","Type":"ContainerDied","Data":"0f903b5badb3d175140635265c6dfd3f22633eaf651ea6ca9e6eac1a8638a5a3"}
Sep 11 01:03:39.705856 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.705675    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f903b5badb3d175140635265c6dfd3f22633eaf651ea6ca9e6eac1a8638a5a3"
Sep 11 01:03:39.707258 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.707231    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn" event={"ID":"44ef9628-d7d1-4e09-a3b5-9b9e7d9df427","Type":"ContainerStarted","Data":"59a8eac4e681645fa49b3c02f5767aafd84dbc3b584fafc65247c3925d3cceb6"}
Sep 11 01:03:39.724586 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:39.724545    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k7ljn" podStartSLOduration=1.1045694560000001 podStartE2EDuration="3.724530097s" totalImagesPullingTime="2.619960641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:03:39.723486701 +0000 UTC m=+339.238086886" watchObservedRunningTime="2026-09-11 01:03:39.724530097 +0000 UTC m=+339.239130279"
Sep 11 01:03:42.121287 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.121242    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hl6fh"]
Sep 11 01:03:42.121686 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.121648    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd635b7-4f9f-4c62-9914-54c81aa58ac8" containerName="util"
Sep 11 01:03:42.121686 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.121661    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd635b7-4f9f-4c62-9914-54c81aa58ac8" containerName="util"
Sep 11 01:03:42.121686 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.121677    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd635b7-4f9f-4c62-9914-54c81aa58ac8" containerName="pull"
Sep 11 01:03:42.121686 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.121682    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd635b7-4f9f-4c62-9914-54c81aa58ac8" containerName="pull"
Sep 11 01:03:42.121822 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.121692    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd635b7-4f9f-4c62-9914-54c81aa58ac8" containerName="extract"
Sep 11 01:03:42.121822 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.121698    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd635b7-4f9f-4c62-9914-54c81aa58ac8" containerName="extract"
Sep 11 01:03:42.121822 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.121757    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dd635b7-4f9f-4c62-9914-54c81aa58ac8" containerName="extract"
Sep 11 01:03:42.124173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.124154    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh"
Sep 11 01:03:42.127074 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.127053    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bsx6w\""
Sep 11 01:03:42.136368 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.136341    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hl6fh"]
Sep 11 01:03:42.196586 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.196551    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq6nt\" (UniqueName: \"kubernetes.io/projected/8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba-kube-api-access-wq6nt\") pod \"cert-manager-webhook-6486ffd997-hl6fh\" (UID: \"8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh"
Sep 11 01:03:42.196765 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.196711    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hl6fh\" (UID: \"8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh"
Sep 11 01:03:42.297803 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.297758    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hl6fh\" (UID: \"8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh"
Sep 11 01:03:42.298009 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.297831    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq6nt\" (UniqueName: \"kubernetes.io/projected/8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba-kube-api-access-wq6nt\") pod \"cert-manager-webhook-6486ffd997-hl6fh\" (UID: \"8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh"
Sep 11 01:03:42.306130 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.306092    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hl6fh\" (UID: \"8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh"
Sep 11 01:03:42.306382 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.306361    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq6nt\" (UniqueName: \"kubernetes.io/projected/8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba-kube-api-access-wq6nt\") pod \"cert-manager-webhook-6486ffd997-hl6fh\" (UID: \"8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh"
Sep 11 01:03:42.433500 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.433409    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh"
Sep 11 01:03:42.703178 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.703153    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hl6fh"]
Sep 11 01:03:42.705206 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:03:42.705177    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e98bfeb_48b4_45b5_87a4_9ca461b9e9ba.slice/crio-c3df4d9a94994caf2796c393ef608dc50b6db7f5b5a53bc6e00a82d8ca1a011e WatchSource:0}: Error finding container c3df4d9a94994caf2796c393ef608dc50b6db7f5b5a53bc6e00a82d8ca1a011e: Status 404 returned error can't find the container with id c3df4d9a94994caf2796c393ef608dc50b6db7f5b5a53bc6e00a82d8ca1a011e
Sep 11 01:03:42.720886 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:42.720858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh" event={"ID":"8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba","Type":"ContainerStarted","Data":"c3df4d9a94994caf2796c393ef608dc50b6db7f5b5a53bc6e00a82d8ca1a011e"}
Sep 11 01:03:43.727202 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:43.727168    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh" event={"ID":"8e98bfeb-48b4-45b5-87a4-9ca461b9e9ba","Type":"ContainerStarted","Data":"c78cee46849c01b8ab6a00c422ab3b9dc37d85d7352b99ed815ac19e676275e1"}
Sep 11 01:03:43.727673 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:43.727285    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh"
Sep 11 01:03:43.745048 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:43.744997    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh" podStartSLOduration=1.744980325 podStartE2EDuration="1.744980325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:03:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:03:43.743674883 +0000 UTC m=+343.258275068" watchObservedRunningTime="2026-09-11 01:03:43.744980325 +0000 UTC m=+343.259580509"
Sep 11 01:03:44.224464 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.224432    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hj52t"]
Sep 11 01:03:44.226689 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.226595    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hj52t"
Sep 11 01:03:44.229253 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.229232    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-6vpnn\""
Sep 11 01:03:44.236327 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.236300    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hj52t"]
Sep 11 01:03:44.309869 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.309836    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbl88\" (UniqueName: \"kubernetes.io/projected/afee6afd-d11a-489e-a007-14299b3031ea-kube-api-access-zbl88\") pod \"cert-manager-55b4d954d5-hj52t\" (UID: \"afee6afd-d11a-489e-a007-14299b3031ea\") " pod="cert-manager/cert-manager-55b4d954d5-hj52t"
Sep 11 01:03:44.310012 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.309886    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/afee6afd-d11a-489e-a007-14299b3031ea-bound-sa-token\") pod \"cert-manager-55b4d954d5-hj52t\" (UID: \"afee6afd-d11a-489e-a007-14299b3031ea\") " pod="cert-manager/cert-manager-55b4d954d5-hj52t"
Sep 11 01:03:44.410823 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.410787    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbl88\" (UniqueName: \"kubernetes.io/projected/afee6afd-d11a-489e-a007-14299b3031ea-kube-api-access-zbl88\") pod \"cert-manager-55b4d954d5-hj52t\" (UID: \"afee6afd-d11a-489e-a007-14299b3031ea\") " pod="cert-manager/cert-manager-55b4d954d5-hj52t"
Sep 11 01:03:44.411017 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.410841    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/afee6afd-d11a-489e-a007-14299b3031ea-bound-sa-token\") pod \"cert-manager-55b4d954d5-hj52t\" (UID: \"afee6afd-d11a-489e-a007-14299b3031ea\") " pod="cert-manager/cert-manager-55b4d954d5-hj52t"
Sep 11 01:03:44.420754 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.420725    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/afee6afd-d11a-489e-a007-14299b3031ea-bound-sa-token\") pod \"cert-manager-55b4d954d5-hj52t\" (UID: \"afee6afd-d11a-489e-a007-14299b3031ea\") " pod="cert-manager/cert-manager-55b4d954d5-hj52t"
Sep 11 01:03:44.421001 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.420979    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbl88\" (UniqueName: \"kubernetes.io/projected/afee6afd-d11a-489e-a007-14299b3031ea-kube-api-access-zbl88\") pod \"cert-manager-55b4d954d5-hj52t\" (UID: \"afee6afd-d11a-489e-a007-14299b3031ea\") " pod="cert-manager/cert-manager-55b4d954d5-hj52t"
Sep 11 01:03:44.536417 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.536389    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hj52t"
Sep 11 01:03:44.805362 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:44.805283    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hj52t"]
Sep 11 01:03:44.808273 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:03:44.808246    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafee6afd_d11a_489e_a007_14299b3031ea.slice/crio-31f8e90c708754f4da016210d41eeea605eea293fddb4e224febaf2a5f037c76 WatchSource:0}: Error finding container 31f8e90c708754f4da016210d41eeea605eea293fddb4e224febaf2a5f037c76: Status 404 returned error can't find the container with id 31f8e90c708754f4da016210d41eeea605eea293fddb4e224febaf2a5f037c76
Sep 11 01:03:45.738055 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:45.738013    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hj52t" event={"ID":"afee6afd-d11a-489e-a007-14299b3031ea","Type":"ContainerStarted","Data":"e75735b693b65624295e9280d9c7540e045d8c522f5d096bff4b37ca3b69d5ef"}
Sep 11 01:03:45.738247 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:45.738063    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hj52t" event={"ID":"afee6afd-d11a-489e-a007-14299b3031ea","Type":"ContainerStarted","Data":"31f8e90c708754f4da016210d41eeea605eea293fddb4e224febaf2a5f037c76"}
Sep 11 01:03:45.755717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:45.755671    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-hj52t" podStartSLOduration=1.75565454 podStartE2EDuration="1.75565454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:03:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:03:45.754307863 +0000 UTC m=+345.268908036" watchObservedRunningTime="2026-09-11 01:03:45.75565454 +0000 UTC m=+345.270254721"
Sep 11 01:03:49.734188 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:49.734153    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hl6fh"
Sep 11 01:03:52.759849 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.759813    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"]
Sep 11 01:03:52.807309 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.807272    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"]
Sep 11 01:03:52.807471 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.807376    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:52.810463 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.810433    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 01:03:52.810584 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.810433    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w87xq\""
Sep 11 01:03:52.810584 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.810478    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 01:03:52.881011 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.880973    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:52.881182 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.881092    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg6x7\" (UniqueName: \"kubernetes.io/projected/827cd2af-9206-482d-82a9-cf67f4213a09-kube-api-access-gg6x7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:52.881182 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.881129    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:52.982421 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.982339    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg6x7\" (UniqueName: \"kubernetes.io/projected/827cd2af-9206-482d-82a9-cf67f4213a09-kube-api-access-gg6x7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:52.982421 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.982390    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:52.982690 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.982427    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:52.982877 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.982854    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:52.982945 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:52.982884    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:53.000069 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:53.000043    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg6x7\" (UniqueName: \"kubernetes.io/projected/827cd2af-9206-482d-82a9-cf67f4213a09-kube-api-access-gg6x7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:53.116804 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:53.116778    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:53.393400 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:53.393248    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"]
Sep 11 01:03:53.396310 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:03:53.396279    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod827cd2af_9206_482d_82a9_cf67f4213a09.slice/crio-1f6740bb231d25abe97d63e644b90caf26d7cde7f362d238fbe8c613a254804a WatchSource:0}: Error finding container 1f6740bb231d25abe97d63e644b90caf26d7cde7f362d238fbe8c613a254804a: Status 404 returned error can't find the container with id 1f6740bb231d25abe97d63e644b90caf26d7cde7f362d238fbe8c613a254804a
Sep 11 01:03:53.774957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:53.774921    2559 generic.go:356] "Generic (PLEG): container finished" podID="827cd2af-9206-482d-82a9-cf67f4213a09" containerID="80bc0cedff96c9f19b64db0a454b0232a9b6b985abd686689a1c4be1e5100b66" exitCode=0
Sep 11 01:03:53.775343 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:53.775008    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd" event={"ID":"827cd2af-9206-482d-82a9-cf67f4213a09","Type":"ContainerDied","Data":"80bc0cedff96c9f19b64db0a454b0232a9b6b985abd686689a1c4be1e5100b66"}
Sep 11 01:03:53.775343 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:53.775058    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd" event={"ID":"827cd2af-9206-482d-82a9-cf67f4213a09","Type":"ContainerStarted","Data":"1f6740bb231d25abe97d63e644b90caf26d7cde7f362d238fbe8c613a254804a"}
Sep 11 01:03:54.781829 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:54.781794    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd" event={"ID":"827cd2af-9206-482d-82a9-cf67f4213a09","Type":"ContainerStarted","Data":"56a3343baa2849cb6e3136f5317ff526faf67b8c9a8da32a9d102c9c8c025d9f"}
Sep 11 01:03:55.788265 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:55.788234    2559 generic.go:356] "Generic (PLEG): container finished" podID="827cd2af-9206-482d-82a9-cf67f4213a09" containerID="56a3343baa2849cb6e3136f5317ff526faf67b8c9a8da32a9d102c9c8c025d9f" exitCode=0
Sep 11 01:03:55.788652 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:55.788315    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd" event={"ID":"827cd2af-9206-482d-82a9-cf67f4213a09","Type":"ContainerDied","Data":"56a3343baa2849cb6e3136f5317ff526faf67b8c9a8da32a9d102c9c8c025d9f"}
Sep 11 01:03:56.797858 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:56.797822    2559 generic.go:356] "Generic (PLEG): container finished" podID="827cd2af-9206-482d-82a9-cf67f4213a09" containerID="89d4dce72b1a11fbf0aaffe59250ed68e0cf2746451d98504fd2d624d1b0684a" exitCode=0
Sep 11 01:03:56.798211 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:56.797902    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd" event={"ID":"827cd2af-9206-482d-82a9-cf67f4213a09","Type":"ContainerDied","Data":"89d4dce72b1a11fbf0aaffe59250ed68e0cf2746451d98504fd2d624d1b0684a"}
Sep 11 01:03:57.922865 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:57.922844    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:58.020359 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.020320    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg6x7\" (UniqueName: \"kubernetes.io/projected/827cd2af-9206-482d-82a9-cf67f4213a09-kube-api-access-gg6x7\") pod \"827cd2af-9206-482d-82a9-cf67f4213a09\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") "
Sep 11 01:03:58.020531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.020425    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-bundle\") pod \"827cd2af-9206-482d-82a9-cf67f4213a09\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") "
Sep 11 01:03:58.020531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.020450    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-util\") pod \"827cd2af-9206-482d-82a9-cf67f4213a09\" (UID: \"827cd2af-9206-482d-82a9-cf67f4213a09\") "
Sep 11 01:03:58.021186 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.021156    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-bundle" (OuterVolumeSpecName: "bundle") pod "827cd2af-9206-482d-82a9-cf67f4213a09" (UID: "827cd2af-9206-482d-82a9-cf67f4213a09"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:03:58.022430 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.022407    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/827cd2af-9206-482d-82a9-cf67f4213a09-kube-api-access-gg6x7" (OuterVolumeSpecName: "kube-api-access-gg6x7") pod "827cd2af-9206-482d-82a9-cf67f4213a09" (UID: "827cd2af-9206-482d-82a9-cf67f4213a09"). InnerVolumeSpecName "kube-api-access-gg6x7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:03:58.028541 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.028515    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-util" (OuterVolumeSpecName: "util") pod "827cd2af-9206-482d-82a9-cf67f4213a09" (UID: "827cd2af-9206-482d-82a9-cf67f4213a09"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:03:58.121445 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.121347    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-util\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:03:58.121445 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.121390    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg6x7\" (UniqueName: \"kubernetes.io/projected/827cd2af-9206-482d-82a9-cf67f4213a09-kube-api-access-gg6x7\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:03:58.121445 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.121402    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/827cd2af-9206-482d-82a9-cf67f4213a09-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:03:58.808682 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.808656    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd"
Sep 11 01:03:58.808850 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.808657    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af862rd" event={"ID":"827cd2af-9206-482d-82a9-cf67f4213a09","Type":"ContainerDied","Data":"1f6740bb231d25abe97d63e644b90caf26d7cde7f362d238fbe8c613a254804a"}
Sep 11 01:03:58.808850 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:03:58.808774    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f6740bb231d25abe97d63e644b90caf26d7cde7f362d238fbe8c613a254804a"
Sep 11 01:04:08.770214 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.770179    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"]
Sep 11 01:04:08.770628 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.770487    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="827cd2af-9206-482d-82a9-cf67f4213a09" containerName="extract"
Sep 11 01:04:08.770628 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.770497    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="827cd2af-9206-482d-82a9-cf67f4213a09" containerName="extract"
Sep 11 01:04:08.770628 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.770514    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="827cd2af-9206-482d-82a9-cf67f4213a09" containerName="util"
Sep 11 01:04:08.770628 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.770519    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="827cd2af-9206-482d-82a9-cf67f4213a09" containerName="util"
Sep 11 01:04:08.770628 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.770533    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="827cd2af-9206-482d-82a9-cf67f4213a09" containerName="pull"
Sep 11 01:04:08.770628 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.770538    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="827cd2af-9206-482d-82a9-cf67f4213a09" containerName="pull"
Sep 11 01:04:08.770628 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.770588    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="827cd2af-9206-482d-82a9-cf67f4213a09" containerName="extract"
Sep 11 01:04:08.773225 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.773207    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:08.776360 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.776344    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w87xq\""
Sep 11 01:04:08.776463 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.776363    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 01:04:08.776463 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.776379    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 01:04:08.782982 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.782960    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"]
Sep 11 01:04:08.916532 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.916492    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:08.916684 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.916544    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnjzv\" (UniqueName: \"kubernetes.io/projected/63db2593-d0ac-4595-9711-1868c53ba727-kube-api-access-pnjzv\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:08.916684 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:08.916632    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:09.017883 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.017844    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:09.018030 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.017888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnjzv\" (UniqueName: \"kubernetes.io/projected/63db2593-d0ac-4595-9711-1868c53ba727-kube-api-access-pnjzv\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:09.018030 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.017908    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:09.018264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.018248    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:09.018307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.018279    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:09.046873 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.046788    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnjzv\" (UniqueName: \"kubernetes.io/projected/63db2593-d0ac-4595-9711-1868c53ba727-kube-api-access-pnjzv\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:09.082529 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.082498    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:09.358635 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.358598    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"]
Sep 11 01:04:09.360709 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:04:09.360686    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63db2593_d0ac_4595_9711_1868c53ba727.slice/crio-409ea3bba28b71a99902574b6b6a6436a3c0d863c3ead706b963675fcbbbf7ba WatchSource:0}: Error finding container 409ea3bba28b71a99902574b6b6a6436a3c0d863c3ead706b963675fcbbbf7ba: Status 404 returned error can't find the container with id 409ea3bba28b71a99902574b6b6a6436a3c0d863c3ead706b963675fcbbbf7ba
Sep 11 01:04:09.860695 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.860659    2559 generic.go:356] "Generic (PLEG): container finished" podID="63db2593-d0ac-4595-9711-1868c53ba727" containerID="577e919b7265a2e1bbed577f8a4cd5b8e4bf484850d1de25905e1eb553d0341f" exitCode=0
Sep 11 01:04:09.861110 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.860747    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l" event={"ID":"63db2593-d0ac-4595-9711-1868c53ba727","Type":"ContainerDied","Data":"577e919b7265a2e1bbed577f8a4cd5b8e4bf484850d1de25905e1eb553d0341f"}
Sep 11 01:04:09.861110 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:09.860804    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l" event={"ID":"63db2593-d0ac-4595-9711-1868c53ba727","Type":"ContainerStarted","Data":"409ea3bba28b71a99902574b6b6a6436a3c0d863c3ead706b963675fcbbbf7ba"}
Sep 11 01:04:10.165581 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.165499    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"]
Sep 11 01:04:10.168135 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.168117    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.172695 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.172674    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 01:04:10.172898 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.172884    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 01:04:10.172965 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.172914    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-h5l9p\""
Sep 11 01:04:10.173307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.173286    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 01:04:10.173443 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.173326    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 01:04:10.183443 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.183422    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"]
Sep 11 01:04:10.230141 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.230101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-manifests\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.230335 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.230207    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbfcm\" (UniqueName: \"kubernetes.io/projected/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-kube-api-access-qbfcm\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.230335 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.230251    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.230335 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.230287    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.230513 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.230425    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-tmp\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.331588 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.331562    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-tmp\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.331715 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.331624    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-manifests\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.331715 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.331664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbfcm\" (UniqueName: \"kubernetes.io/projected/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-kube-api-access-qbfcm\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.331715 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.331685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.331715 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.331710    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.332016 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.331998    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-manifests\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.333814 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.333791    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-tmp\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.334094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.334075    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.334173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.334133    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-webhook-cert\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.340797 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.340776    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbfcm\" (UniqueName: \"kubernetes.io/projected/4c001ce3-70a0-427f-8da3-8242f5f7ceb7-kube-api-access-qbfcm\") pod \"opendatahub-operator-controller-manager-8bd47c7d5-qkzmm\" (UID: \"4c001ce3-70a0-427f-8da3-8242f5f7ceb7\") " pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.478210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.478137    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:10.754544 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.754517    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"]
Sep 11 01:04:10.756655 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:04:10.756626    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c001ce3_70a0_427f_8da3_8242f5f7ceb7.slice/crio-9e28940f084ec192a78f26be264d4740be357718884e83bf5f2db4824ca98b76 WatchSource:0}: Error finding container 9e28940f084ec192a78f26be264d4740be357718884e83bf5f2db4824ca98b76: Status 404 returned error can't find the container with id 9e28940f084ec192a78f26be264d4740be357718884e83bf5f2db4824ca98b76
Sep 11 01:04:10.867342 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.867302    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm" event={"ID":"4c001ce3-70a0-427f-8da3-8242f5f7ceb7","Type":"ContainerStarted","Data":"9e28940f084ec192a78f26be264d4740be357718884e83bf5f2db4824ca98b76"}
Sep 11 01:04:10.869071 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.869046    2559 generic.go:356] "Generic (PLEG): container finished" podID="63db2593-d0ac-4595-9711-1868c53ba727" containerID="23880e496f4f4cfee7731282c00f1106d16747090700bbe4575d6d37133f6f0b" exitCode=0
Sep 11 01:04:10.869171 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:10.869135    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l" event={"ID":"63db2593-d0ac-4595-9711-1868c53ba727","Type":"ContainerDied","Data":"23880e496f4f4cfee7731282c00f1106d16747090700bbe4575d6d37133f6f0b"}
Sep 11 01:04:11.486583 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.486362    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57655f9cf9-jh6r8"]
Sep 11 01:04:11.489375 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.489354    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.507520 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.503327    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57655f9cf9-jh6r8"]
Sep 11 01:04:11.644848 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.644801    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-console-config\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.645028 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.644876    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-oauth-serving-cert\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.645028 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.644928    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5ghd\" (UniqueName: \"kubernetes.io/projected/8ed02274-604a-46cb-a096-38c281ec478b-kube-api-access-v5ghd\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.645028 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.644954    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-service-ca\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.645028 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.645006    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-trusted-ca-bundle\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.645270 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.645039    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ed02274-604a-46cb-a096-38c281ec478b-console-serving-cert\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.645270 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.645062    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ed02274-604a-46cb-a096-38c281ec478b-console-oauth-config\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.746565 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.746351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5ghd\" (UniqueName: \"kubernetes.io/projected/8ed02274-604a-46cb-a096-38c281ec478b-kube-api-access-v5ghd\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.746565 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.746406    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-service-ca\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.746565 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.746461    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-trusted-ca-bundle\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.746565 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.746495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ed02274-604a-46cb-a096-38c281ec478b-console-serving-cert\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.746565 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.746520    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ed02274-604a-46cb-a096-38c281ec478b-console-oauth-config\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.746975 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.746581    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-console-config\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.747559 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.747213    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-oauth-serving-cert\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.747749 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.747578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-service-ca\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.747749 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.747703    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-trusted-ca-bundle\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.748291 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.748260    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-console-config\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.748883 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.748840    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ed02274-604a-46cb-a096-38c281ec478b-oauth-serving-cert\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.750091 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.750067    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ed02274-604a-46cb-a096-38c281ec478b-console-oauth-config\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.750239 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.750217    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ed02274-604a-46cb-a096-38c281ec478b-console-serving-cert\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.755886 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.755860    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5ghd\" (UniqueName: \"kubernetes.io/projected/8ed02274-604a-46cb-a096-38c281ec478b-kube-api-access-v5ghd\") pod \"console-57655f9cf9-jh6r8\" (UID: \"8ed02274-604a-46cb-a096-38c281ec478b\") " pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.801924 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.801891    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:11.881659 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.881597    2559 generic.go:356] "Generic (PLEG): container finished" podID="63db2593-d0ac-4595-9711-1868c53ba727" containerID="8ed661576665e37d7f5067f827604b180ecd66dd519072be105b0d3e9dd58195" exitCode=0
Sep 11 01:04:11.881996 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:11.881724    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l" event={"ID":"63db2593-d0ac-4595-9711-1868c53ba727","Type":"ContainerDied","Data":"8ed661576665e37d7f5067f827604b180ecd66dd519072be105b0d3e9dd58195"}
Sep 11 01:04:12.089374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:12.089346    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57655f9cf9-jh6r8"]
Sep 11 01:04:12.092327 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:04:12.092290    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed02274_604a_46cb_a096_38c281ec478b.slice/crio-0ab832f72785153f65f49118440f8828f76193e8f7e16d0508aaaab329c59a33 WatchSource:0}: Error finding container 0ab832f72785153f65f49118440f8828f76193e8f7e16d0508aaaab329c59a33: Status 404 returned error can't find the container with id 0ab832f72785153f65f49118440f8828f76193e8f7e16d0508aaaab329c59a33
Sep 11 01:04:12.889655 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:12.889620    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57655f9cf9-jh6r8" event={"ID":"8ed02274-604a-46cb-a096-38c281ec478b","Type":"ContainerStarted","Data":"0ab832f72785153f65f49118440f8828f76193e8f7e16d0508aaaab329c59a33"}
Sep 11 01:04:13.724979 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.724954    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:13.868442 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.868413    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-bundle\") pod \"63db2593-d0ac-4595-9711-1868c53ba727\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") "
Sep 11 01:04:13.868648 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.868501    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-util\") pod \"63db2593-d0ac-4595-9711-1868c53ba727\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") "
Sep 11 01:04:13.868648 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.868567    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnjzv\" (UniqueName: \"kubernetes.io/projected/63db2593-d0ac-4595-9711-1868c53ba727-kube-api-access-pnjzv\") pod \"63db2593-d0ac-4595-9711-1868c53ba727\" (UID: \"63db2593-d0ac-4595-9711-1868c53ba727\") "
Sep 11 01:04:13.869284 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.869257    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-bundle" (OuterVolumeSpecName: "bundle") pod "63db2593-d0ac-4595-9711-1868c53ba727" (UID: "63db2593-d0ac-4595-9711-1868c53ba727"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:04:13.870928 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.870903    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63db2593-d0ac-4595-9711-1868c53ba727-kube-api-access-pnjzv" (OuterVolumeSpecName: "kube-api-access-pnjzv") pod "63db2593-d0ac-4595-9711-1868c53ba727" (UID: "63db2593-d0ac-4595-9711-1868c53ba727"). InnerVolumeSpecName "kube-api-access-pnjzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:04:13.875872 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.875836    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-util" (OuterVolumeSpecName: "util") pod "63db2593-d0ac-4595-9711-1868c53ba727" (UID: "63db2593-d0ac-4595-9711-1868c53ba727"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:04:13.897066 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.897039    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l"
Sep 11 01:04:13.897456 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.897059    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0lfn4l" event={"ID":"63db2593-d0ac-4595-9711-1868c53ba727","Type":"ContainerDied","Data":"409ea3bba28b71a99902574b6b6a6436a3c0d863c3ead706b963675fcbbbf7ba"}
Sep 11 01:04:13.897456 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.897125    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="409ea3bba28b71a99902574b6b6a6436a3c0d863c3ead706b963675fcbbbf7ba"
Sep 11 01:04:13.898868 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.898845    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57655f9cf9-jh6r8" event={"ID":"8ed02274-604a-46cb-a096-38c281ec478b","Type":"ContainerStarted","Data":"ec8b0d489ac41abb895767881a0d884192abf65fb7985340b046def7354fba7c"}
Sep 11 01:04:13.900536 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.900511    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm" event={"ID":"4c001ce3-70a0-427f-8da3-8242f5f7ceb7","Type":"ContainerStarted","Data":"2fe177bc2862abdddf88a72304458aec797fc778c28e802d58dcde93b6ad0fd0"}
Sep 11 01:04:13.917136 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.917065    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57655f9cf9-jh6r8" podStartSLOduration=2.917050219 podStartE2EDuration="2.917050219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:13.915671944 +0000 UTC m=+373.430272126" watchObservedRunningTime="2026-09-11 01:04:13.917050219 +0000 UTC m=+373.431650401"
Sep 11 01:04:13.970541 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.970463    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:13.970541 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.970492    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63db2593-d0ac-4595-9711-1868c53ba727-util\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:13.970541 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:13.970506    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnjzv\" (UniqueName: \"kubernetes.io/projected/63db2593-d0ac-4595-9711-1868c53ba727-kube-api-access-pnjzv\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:14.908037 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:14.908007    2559 generic.go:356] "Generic (PLEG): container finished" podID="4c001ce3-70a0-427f-8da3-8242f5f7ceb7" containerID="2fe177bc2862abdddf88a72304458aec797fc778c28e802d58dcde93b6ad0fd0" exitCode=0
Sep 11 01:04:14.908416 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:14.908083    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm" event={"ID":"4c001ce3-70a0-427f-8da3-8242f5f7ceb7","Type":"ContainerDied","Data":"2fe177bc2862abdddf88a72304458aec797fc778c28e802d58dcde93b6ad0fd0"}
Sep 11 01:04:15.915665 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:15.915631    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm" event={"ID":"4c001ce3-70a0-427f-8da3-8242f5f7ceb7","Type":"ContainerStarted","Data":"48b38f7ceaa5cbc0968c487044af0a6d2b6b4fe5868f382d4b761151db9761df"}
Sep 11 01:04:15.916059 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:15.915856    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:15.951115 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:15.951066    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm" podStartSLOduration=1.598735555 podStartE2EDuration="5.951049894s" totalImagesPullingTime="3.32375699s" totalInitContainerRuntime="1.028557349s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:15.94848981 +0000 UTC m=+375.463090004" watchObservedRunningTime="2026-09-11 01:04:15.951049894 +0000 UTC m=+375.465650087"
Sep 11 01:04:21.802347 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:21.802310    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:21.802827 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:21.802378    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:21.806982 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:21.806960    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:21.946748 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:21.946720    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57655f9cf9-jh6r8"
Sep 11 01:04:21.994407 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:21.994376    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-549c87c684-d425s"]
Sep 11 01:04:26.921831 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:26.921804    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8bd47c7d5-qkzmm"
Sep 11 01:04:29.745853 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.745812    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"]
Sep 11 01:04:29.746340 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.746321    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63db2593-d0ac-4595-9711-1868c53ba727" containerName="pull"
Sep 11 01:04:29.746419 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.746343    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="63db2593-d0ac-4595-9711-1868c53ba727" containerName="pull"
Sep 11 01:04:29.746419 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.746361    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63db2593-d0ac-4595-9711-1868c53ba727" containerName="extract"
Sep 11 01:04:29.746419 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.746369    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="63db2593-d0ac-4595-9711-1868c53ba727" containerName="extract"
Sep 11 01:04:29.746419 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.746388    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63db2593-d0ac-4595-9711-1868c53ba727" containerName="util"
Sep 11 01:04:29.746419 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.746397    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="63db2593-d0ac-4595-9711-1868c53ba727" containerName="util"
Sep 11 01:04:29.746693 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.746475    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="63db2593-d0ac-4595-9711-1868c53ba727" containerName="extract"
Sep 11 01:04:29.752091 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.752070    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:29.755626 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.755590    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 01:04:29.756301 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.756276    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 01:04:29.757119 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.757104    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 01:04:29.757214 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.757130    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 01:04:29.765736 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.765714    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"]
Sep 11 01:04:29.806196 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.806158    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2854327b-d4ef-4d48-96d2-d30fbf98952f-tmp\") pod \"kube-auth-proxy-58fc6d6988-7j7gd\" (UID: \"2854327b-d4ef-4d48-96d2-d30fbf98952f\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:29.806333 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.806217    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b4jh\" (UniqueName: \"kubernetes.io/projected/2854327b-d4ef-4d48-96d2-d30fbf98952f-kube-api-access-5b4jh\") pod \"kube-auth-proxy-58fc6d6988-7j7gd\" (UID: \"2854327b-d4ef-4d48-96d2-d30fbf98952f\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:29.806333 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.806241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2854327b-d4ef-4d48-96d2-d30fbf98952f-tls-certs\") pod \"kube-auth-proxy-58fc6d6988-7j7gd\" (UID: \"2854327b-d4ef-4d48-96d2-d30fbf98952f\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:29.906957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.906922    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b4jh\" (UniqueName: \"kubernetes.io/projected/2854327b-d4ef-4d48-96d2-d30fbf98952f-kube-api-access-5b4jh\") pod \"kube-auth-proxy-58fc6d6988-7j7gd\" (UID: \"2854327b-d4ef-4d48-96d2-d30fbf98952f\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:29.906957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.906964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2854327b-d4ef-4d48-96d2-d30fbf98952f-tls-certs\") pod \"kube-auth-proxy-58fc6d6988-7j7gd\" (UID: \"2854327b-d4ef-4d48-96d2-d30fbf98952f\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:29.907156 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.907026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2854327b-d4ef-4d48-96d2-d30fbf98952f-tmp\") pod \"kube-auth-proxy-58fc6d6988-7j7gd\" (UID: \"2854327b-d4ef-4d48-96d2-d30fbf98952f\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:29.907156 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:04:29.907137    2559 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 11 01:04:29.907229 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:04:29.907217    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2854327b-d4ef-4d48-96d2-d30fbf98952f-tls-certs podName:2854327b-d4ef-4d48-96d2-d30fbf98952f nodeName:}" failed. No retries permitted until 2026-09-11 01:04:30.407196169 +0000 UTC m=+389.921796338 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2854327b-d4ef-4d48-96d2-d30fbf98952f-tls-certs") pod "kube-auth-proxy-58fc6d6988-7j7gd" (UID: "2854327b-d4ef-4d48-96d2-d30fbf98952f") : secret "kube-auth-proxy-tls" not found
Sep 11 01:04:29.909165 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.909143    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2854327b-d4ef-4d48-96d2-d30fbf98952f-tmp\") pod \"kube-auth-proxy-58fc6d6988-7j7gd\" (UID: \"2854327b-d4ef-4d48-96d2-d30fbf98952f\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:29.934400 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:29.934369    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b4jh\" (UniqueName: \"kubernetes.io/projected/2854327b-d4ef-4d48-96d2-d30fbf98952f-kube-api-access-5b4jh\") pod \"kube-auth-proxy-58fc6d6988-7j7gd\" (UID: \"2854327b-d4ef-4d48-96d2-d30fbf98952f\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:30.410445 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:30.410410    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2854327b-d4ef-4d48-96d2-d30fbf98952f-tls-certs\") pod \"kube-auth-proxy-58fc6d6988-7j7gd\" (UID: \"2854327b-d4ef-4d48-96d2-d30fbf98952f\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:30.412810 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:30.412790    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2854327b-d4ef-4d48-96d2-d30fbf98952f-tls-certs\") pod \"kube-auth-proxy-58fc6d6988-7j7gd\" (UID: \"2854327b-d4ef-4d48-96d2-d30fbf98952f\") " pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:30.661629 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:30.661535    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"
Sep 11 01:04:30.932557 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:30.932534    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd"]
Sep 11 01:04:30.934462 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:04:30.934429    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2854327b_d4ef_4d48_96d2_d30fbf98952f.slice/crio-c40e521a77b86216877c000dd7f4ac8d58804dbf1790199470124b190b90964e WatchSource:0}: Error finding container c40e521a77b86216877c000dd7f4ac8d58804dbf1790199470124b190b90964e: Status 404 returned error can't find the container with id c40e521a77b86216877c000dd7f4ac8d58804dbf1790199470124b190b90964e
Sep 11 01:04:30.990843 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:30.990805    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd" event={"ID":"2854327b-d4ef-4d48-96d2-d30fbf98952f","Type":"ContainerStarted","Data":"c40e521a77b86216877c000dd7f4ac8d58804dbf1790199470124b190b90964e"}
Sep 11 01:04:34.016942 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:34.016903    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd" event={"ID":"2854327b-d4ef-4d48-96d2-d30fbf98952f","Type":"ContainerStarted","Data":"dc11656c06a209e04d92668440d68c40a93a45712d135bf26219eb518efa082f"}
Sep 11 01:04:34.033918 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:34.033862    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58fc6d6988-7j7gd" podStartSLOduration=2.325497692 podStartE2EDuration="5.033845217s" totalImagesPullingTime="2.708347525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:34.03286489 +0000 UTC m=+393.547465073" watchObservedRunningTime="2026-09-11 01:04:34.033845217 +0000 UTC m=+393.548445411"
Sep 11 01:04:37.052456 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.052414    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"]
Sep 11 01:04:37.056380 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.056363    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.059027 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.059004    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 01:04:37.059266 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.059250    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4khcg\""
Sep 11 01:04:37.065457 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.065433    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"]
Sep 11 01:04:37.165575 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.165537    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.165772 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.165576    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c45kh\" (UniqueName: \"kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-kube-api-access-c45kh\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.165772 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.165636    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.165772 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.165670    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.165772 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.165738    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.165986 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.165855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.165986 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.165914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.165986 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.165949    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9cea652-a164-4d8f-829e-8277e4e1332f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.166134 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.165993    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.204504 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.204471    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"]
Sep 11 01:04:37.208102 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.208081    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.219414 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.219390    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"]
Sep 11 01:04:37.266556 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266519    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.266556 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.266809 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.266809 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266597    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.266809 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266643    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9cea652-a164-4d8f-829e-8277e4e1332f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.266809 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266665    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crhbc\" (UniqueName: \"kubernetes.io/projected/855298c1-828a-44c3-8656-727732d1722c-kube-api-access-crhbc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.266809 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266699    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.266809 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266731    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/855298c1-828a-44c3-8656-727732d1722c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.267121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266877    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/855298c1-828a-44c3-8656-727732d1722c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.267121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.267121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.266972    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.267121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267006    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.267121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267015    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45kh\" (UniqueName: \"kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-kube-api-access-c45kh\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.267121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267066    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.267121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267096    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.267461 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.267461 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267164    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.267461 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267190    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/855298c1-828a-44c3-8656-727732d1722c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.267461 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267238    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.267461 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267260    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.267461 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267329    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.267461 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267340    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9cea652-a164-4d8f-829e-8277e4e1332f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.267461 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.267452    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.269156 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.269138    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.269401 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.269384    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.278153 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.278134    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.278470 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.278446    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c45kh\" (UniqueName: \"kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-kube-api-access-c45kh\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.368119 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368034    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:37.368258 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368153    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368258 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368183    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/855298c1-828a-44c3-8656-727732d1722c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368258 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368215    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368258 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368233    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368438 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368263    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368438 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368301    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crhbc\" (UniqueName: \"kubernetes.io/projected/855298c1-828a-44c3-8656-727732d1722c-kube-api-access-crhbc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368438 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/855298c1-828a-44c3-8656-727732d1722c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368438 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368371    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/855298c1-828a-44c3-8656-727732d1722c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368438 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368405    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368733 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368595    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368789 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368732    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.368841 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368796    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.369001 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.368978    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.369054 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.369026    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/855298c1-828a-44c3-8656-727732d1722c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.370576 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.370556    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/855298c1-828a-44c3-8656-727732d1722c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.370897 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.370880    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/855298c1-828a-44c3-8656-727732d1722c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.375853 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.375832    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/855298c1-828a-44c3-8656-727732d1722c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.376050 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.376034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crhbc\" (UniqueName: \"kubernetes.io/projected/855298c1-828a-44c3-8656-727732d1722c-kube-api-access-crhbc\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn\" (UID: \"855298c1-828a-44c3-8656-727732d1722c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.518023 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.517988    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:37.641235 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.641210    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"]
Sep 11 01:04:37.642556 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:04:37.642534    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9cea652_a164_4d8f_829e_8277e4e1332f.slice/crio-46fd1f35cb646dc8f9d713b07a5d5449b279da028bf7491c8fd11ab9571bd5b8 WatchSource:0}: Error finding container 46fd1f35cb646dc8f9d713b07a5d5449b279da028bf7491c8fd11ab9571bd5b8: Status 404 returned error can't find the container with id 46fd1f35cb646dc8f9d713b07a5d5449b279da028bf7491c8fd11ab9571bd5b8
Sep 11 01:04:37.792039 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:37.792012    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"]
Sep 11 01:04:37.793650 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:04:37.793627    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855298c1_828a_44c3_8656_727732d1722c.slice/crio-2269d1a130b8e0b087c1825a059772cd46df4bb510d4d79634af0c14fac6ad61 WatchSource:0}: Error finding container 2269d1a130b8e0b087c1825a059772cd46df4bb510d4d79634af0c14fac6ad61: Status 404 returned error can't find the container with id 2269d1a130b8e0b087c1825a059772cd46df4bb510d4d79634af0c14fac6ad61
Sep 11 01:04:38.035262 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:38.035227    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6" event={"ID":"e9cea652-a164-4d8f-829e-8277e4e1332f","Type":"ContainerStarted","Data":"46fd1f35cb646dc8f9d713b07a5d5449b279da028bf7491c8fd11ab9571bd5b8"}
Sep 11 01:04:38.036425 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:38.036404    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn" event={"ID":"855298c1-828a-44c3-8656-727732d1722c","Type":"ContainerStarted","Data":"2269d1a130b8e0b087c1825a059772cd46df4bb510d4d79634af0c14fac6ad61"}
Sep 11 01:04:40.286052 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:40.286008    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:04:40.286365 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:40.286085    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:04:40.286365 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:40.286121    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:04:40.292516 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:40.292480    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:04:40.292598 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:40.292554    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:04:40.292598 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:40.292587    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:04:41.054254 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:41.054216    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6" event={"ID":"e9cea652-a164-4d8f-829e-8277e4e1332f","Type":"ContainerStarted","Data":"69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba"}
Sep 11 01:04:41.055734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:41.055713    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn" event={"ID":"855298c1-828a-44c3-8656-727732d1722c","Type":"ContainerStarted","Data":"2d94d17c3566ce01b6b2ea4b062e6fb5fdabbb0a908ed7a1e4fcb1a4065b3199"}
Sep 11 01:04:41.077890 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:41.077848    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6" podStartSLOduration=1.436671255 podStartE2EDuration="4.077834995s" totalImagesPullingTime="2.64116374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:41.076916045 +0000 UTC m=+400.591516227" watchObservedRunningTime="2026-09-11 01:04:41.077834995 +0000 UTC m=+400.592435181"
Sep 11 01:04:41.097565 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:41.097527    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn" podStartSLOduration=1.600892094 podStartE2EDuration="4.097514321s" totalImagesPullingTime="2.496622227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:41.095355514 +0000 UTC m=+400.609955698" watchObservedRunningTime="2026-09-11 01:04:41.097514321 +0000 UTC m=+400.612114514"
Sep 11 01:04:41.369226 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:41.369134    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:41.370556 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:41.370527    2559 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 11 01:04:41.370682 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:41.370593    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6" podUID="e9cea652-a164-4d8f-829e-8277e4e1332f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 11 01:04:41.519192 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:41.519157    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:41.524070 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:41.524047    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:42.060649 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:42.060599    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:42.061528 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:42.061507    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn"
Sep 11 01:04:42.117128 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:42.117080    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"]
Sep 11 01:04:42.369521 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:42.369438    2559 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 11 01:04:42.369900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:42.369516    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6" podUID="e9cea652-a164-4d8f-829e-8277e4e1332f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 11 01:04:43.369177 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:43.369134    2559 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 11 01:04:43.369391 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:43.369214    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6" podUID="e9cea652-a164-4d8f-829e-8277e4e1332f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 11 01:04:44.070183 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:44.070148    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6" podUID="e9cea652-a164-4d8f-829e-8277e4e1332f" containerName="istio-proxy" containerID="cri-o://69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba" gracePeriod=30
Sep 11 01:04:47.016066 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.016028    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-549c87c684-d425s" podUID="d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" containerName="console" containerID="cri-o://f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9" gracePeriod=15
Sep 11 01:04:47.264247 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.264222    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:04:47.365256 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365175    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-service-ca\") pod \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") "
Sep 11 01:04:47.365256 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365214    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-config\") pod \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") "
Sep 11 01:04:47.365487 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365273    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqvx6\" (UniqueName: \"kubernetes.io/projected/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-kube-api-access-sqvx6\") pod \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") "
Sep 11 01:04:47.365487 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365295    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-serving-cert\") pod \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") "
Sep 11 01:04:47.365487 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365342    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-oauth-config\") pod \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") "
Sep 11 01:04:47.365487 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365408    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-oauth-serving-cert\") pod \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") "
Sep 11 01:04:47.365719 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365493    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-trusted-ca-bundle\") pod \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\" (UID: \"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c\") "
Sep 11 01:04:47.365719 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365630    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-service-ca" (OuterVolumeSpecName: "service-ca") pod "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" (UID: "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:04:47.365819 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365769    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-config" (OuterVolumeSpecName: "console-config") pod "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" (UID: "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:04:47.365860 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365811    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" (UID: "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:04:47.365915 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365852    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:47.365915 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.365872    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:47.366058 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.366019    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" (UID: "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:04:47.367459 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.367437    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-kube-api-access-sqvx6" (OuterVolumeSpecName: "kube-api-access-sqvx6") pod "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" (UID: "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c"). InnerVolumeSpecName "kube-api-access-sqvx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:04:47.367910 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.367889    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" (UID: "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:04:47.367982 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.367905    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" (UID: "d2111085-c0b7-4a9c-9cd1-44ef07adaf9c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:04:47.467185 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.467153    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqvx6\" (UniqueName: \"kubernetes.io/projected/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-kube-api-access-sqvx6\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:47.467185 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.467184    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-serving-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:47.467375 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.467195    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-console-oauth-config\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:47.467375 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.467203    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-oauth-serving-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:47.467375 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:47.467213    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c-trusted-ca-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:48.086812 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.086778    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"]
Sep 11 01:04:48.088147 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.088119    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" containerName="console"
Sep 11 01:04:48.088320 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.088308    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" containerName="console"
Sep 11 01:04:48.088584 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.088572    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" containerName="console"
Sep 11 01:04:48.093740 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.093720    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.097519 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.097495    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 01:04:48.097864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.097515    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ck7ll\""
Sep 11 01:04:48.099985 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.099957    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 01:04:48.100076 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.100030    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 01:04:48.104658 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.103599    2559 generic.go:356] "Generic (PLEG): container finished" podID="d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" containerID="f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9" exitCode=0
Sep 11 01:04:48.104658 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.103776    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549c87c684-d425s" event={"ID":"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c","Type":"ContainerDied","Data":"f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9"}
Sep 11 01:04:48.104658 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.103814    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549c87c684-d425s" event={"ID":"d2111085-c0b7-4a9c-9cd1-44ef07adaf9c","Type":"ContainerDied","Data":"316ac4c70f67bf429530ad0956f30a33e1817976ab3807b8e04945aa238c53d9"}
Sep 11 01:04:48.104658 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.103841    2559 scope.go:117] "RemoveContainer" containerID="f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9"
Sep 11 01:04:48.104658 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.104014    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549c87c684-d425s"
Sep 11 01:04:48.104658 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.104593    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"]
Sep 11 01:04:48.114264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.114219    2559 scope.go:117] "RemoveContainer" containerID="f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9"
Sep 11 01:04:48.114681 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:04:48.114661    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9\": container with ID starting with f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9 not found: ID does not exist" containerID="f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9"
Sep 11 01:04:48.114760 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.114689    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9"} err="failed to get container status \"f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9\": rpc error: code = NotFound desc = could not find container \"f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9\": container with ID starting with f221e3657a49b198eb08fbdb98de8c82fb59e31f6ceae4bcd0151073887986b9 not found: ID does not exist"
Sep 11 01:04:48.145931 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.145898    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-549c87c684-d425s"]
Sep 11 01:04:48.153341 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.153313    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-549c87c684-d425s"]
Sep 11 01:04:48.172490 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.172460    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxc5s\" (UniqueName: \"kubernetes.io/projected/0b19cad6-8942-43ea-aa5f-01d6bba6f992-kube-api-access-pxc5s\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.172594 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.172503    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.172594 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.172551    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.172715 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.172650    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.172715 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.172680    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b19cad6-8942-43ea-aa5f-01d6bba6f992-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.172715 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.172702    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b19cad6-8942-43ea-aa5f-01d6bba6f992-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.172825 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.172805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.274069 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.274032    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.274427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.274086    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.274427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.274119    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b19cad6-8942-43ea-aa5f-01d6bba6f992-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.274427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.274146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b19cad6-8942-43ea-aa5f-01d6bba6f992-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.274427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.274197    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.274427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.274230    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxc5s\" (UniqueName: \"kubernetes.io/projected/0b19cad6-8942-43ea-aa5f-01d6bba6f992-kube-api-access-pxc5s\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.274427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.274264    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.274911 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.274889    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.276709 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.276680    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0b19cad6-8942-43ea-aa5f-01d6bba6f992-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.276886 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.276865    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.276953 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.276868    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b19cad6-8942-43ea-aa5f-01d6bba6f992-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.276990 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.276974    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.297423 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.297394    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b19cad6-8942-43ea-aa5f-01d6bba6f992-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.297846 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.297826    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxc5s\" (UniqueName: \"kubernetes.io/projected/0b19cad6-8942-43ea-aa5f-01d6bba6f992-kube-api-access-pxc5s\") pod \"istiod-openshift-gateway-5677ccb49b-qrb6l\" (UID: \"0b19cad6-8942-43ea-aa5f-01d6bba6f992\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.407217 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.407137    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:48.711976 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:48.711950    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"]
Sep 11 01:04:48.713340 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:04:48.713313    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b19cad6_8942_43ea_aa5f_01d6bba6f992.slice/crio-11724155e7ff08bc9897455ee2f9d8adad73df2ad7dc3a010625e470a7a9c6d3 WatchSource:0}: Error finding container 11724155e7ff08bc9897455ee2f9d8adad73df2ad7dc3a010625e470a7a9c6d3: Status 404 returned error can't find the container with id 11724155e7ff08bc9897455ee2f9d8adad73df2ad7dc3a010625e470a7a9c6d3
Sep 11 01:04:49.109667 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.109633    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2111085-c0b7-4a9c-9cd1-44ef07adaf9c" path="/var/lib/kubelet/pods/d2111085-c0b7-4a9c-9cd1-44ef07adaf9c/volumes"
Sep 11 01:04:49.110461 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.110438    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l" event={"ID":"0b19cad6-8942-43ea-aa5f-01d6bba6f992","Type":"ContainerStarted","Data":"11724155e7ff08bc9897455ee2f9d8adad73df2ad7dc3a010625e470a7a9c6d3"}
Sep 11 01:04:49.308387 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.308361    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:49.388407 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388336    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-token\") pod \"e9cea652-a164-4d8f-829e-8277e4e1332f\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") "
Sep 11 01:04:49.388407 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388404    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9cea652-a164-4d8f-829e-8277e4e1332f-istiod-ca-cert\") pod \"e9cea652-a164-4d8f-829e-8277e4e1332f\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") "
Sep 11 01:04:49.388597 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388437    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-certs\") pod \"e9cea652-a164-4d8f-829e-8277e4e1332f\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") "
Sep 11 01:04:49.388597 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388477    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-envoy\") pod \"e9cea652-a164-4d8f-829e-8277e4e1332f\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") "
Sep 11 01:04:49.388597 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388504    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c45kh\" (UniqueName: \"kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-kube-api-access-c45kh\") pod \"e9cea652-a164-4d8f-829e-8277e4e1332f\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") "
Sep 11 01:04:49.388597 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388522    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-podinfo\") pod \"e9cea652-a164-4d8f-829e-8277e4e1332f\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") "
Sep 11 01:04:49.388597 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388559    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-data\") pod \"e9cea652-a164-4d8f-829e-8277e4e1332f\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") "
Sep 11 01:04:49.388597 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388584    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-credential-socket\") pod \"e9cea652-a164-4d8f-829e-8277e4e1332f\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") "
Sep 11 01:04:49.388947 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388624    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-socket\") pod \"e9cea652-a164-4d8f-829e-8277e4e1332f\" (UID: \"e9cea652-a164-4d8f-829e-8277e4e1332f\") "
Sep 11 01:04:49.388947 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388783    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e9cea652-a164-4d8f-829e-8277e4e1332f" (UID: "e9cea652-a164-4d8f-829e-8277e4e1332f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:04:49.388947 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388859    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9cea652-a164-4d8f-829e-8277e4e1332f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e9cea652-a164-4d8f-829e-8277e4e1332f" (UID: "e9cea652-a164-4d8f-829e-8277e4e1332f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:04:49.389105 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388947    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9cea652-a164-4d8f-829e-8277e4e1332f-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:49.389105 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.388966    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:49.389105 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.389006    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e9cea652-a164-4d8f-829e-8277e4e1332f" (UID: "e9cea652-a164-4d8f-829e-8277e4e1332f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:04:49.389105 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.389015    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e9cea652-a164-4d8f-829e-8277e4e1332f" (UID: "e9cea652-a164-4d8f-829e-8277e4e1332f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:04:49.389105 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.389062    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e9cea652-a164-4d8f-829e-8277e4e1332f" (UID: "e9cea652-a164-4d8f-829e-8277e4e1332f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:04:49.390708 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.390678    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e9cea652-a164-4d8f-829e-8277e4e1332f" (UID: "e9cea652-a164-4d8f-829e-8277e4e1332f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:04:49.390809 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.390736    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e9cea652-a164-4d8f-829e-8277e4e1332f" (UID: "e9cea652-a164-4d8f-829e-8277e4e1332f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:04:49.390809 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.390762    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e9cea652-a164-4d8f-829e-8277e4e1332f" (UID: "e9cea652-a164-4d8f-829e-8277e4e1332f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:04:49.390896 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.390834    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-kube-api-access-c45kh" (OuterVolumeSpecName: "kube-api-access-c45kh") pod "e9cea652-a164-4d8f-829e-8277e4e1332f" (UID: "e9cea652-a164-4d8f-829e-8277e4e1332f"). InnerVolumeSpecName "kube-api-access-c45kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:04:49.490118 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.490083    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:49.490118 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.490111    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:49.490118 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.490121    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:49.490355 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.490133    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:49.490355 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.490142    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:49.490355 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.490150    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c45kh\" (UniqueName: \"kubernetes.io/projected/e9cea652-a164-4d8f-829e-8277e4e1332f-kube-api-access-c45kh\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:49.490355 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:49.490158    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9cea652-a164-4d8f-829e-8277e4e1332f-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:04:50.119507 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:50.119467    2559 generic.go:356] "Generic (PLEG): container finished" podID="e9cea652-a164-4d8f-829e-8277e4e1332f" containerID="69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba" exitCode=0
Sep 11 01:04:50.119507 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:50.119515    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"
Sep 11 01:04:50.120058 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:50.119643    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6" event={"ID":"e9cea652-a164-4d8f-829e-8277e4e1332f","Type":"ContainerDied","Data":"69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba"}
Sep 11 01:04:50.120058 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:50.119695    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6" event={"ID":"e9cea652-a164-4d8f-829e-8277e4e1332f","Type":"ContainerDied","Data":"46fd1f35cb646dc8f9d713b07a5d5449b279da028bf7491c8fd11ab9571bd5b8"}
Sep 11 01:04:50.120058 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:50.119724    2559 scope.go:117] "RemoveContainer" containerID="69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba"
Sep 11 01:04:50.131531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:50.131512    2559 scope.go:117] "RemoveContainer" containerID="69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba"
Sep 11 01:04:50.131863 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:04:50.131840    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba\": container with ID starting with 69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba not found: ID does not exist" containerID="69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba"
Sep 11 01:04:50.131938 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:50.131873    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba"} err="failed to get container status \"69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba\": rpc error: code = NotFound desc = could not find container \"69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba\": container with ID starting with 69bfab2664cef71d2bc01f3c76d1c72d66328d0f2fee4fd83ddef5c6154425ba not found: ID does not exist"
Sep 11 01:04:50.155104 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:50.155076    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"]
Sep 11 01:04:50.159636 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:50.159613    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bc2sgf6"]
Sep 11 01:04:51.112654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.112624    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9cea652-a164-4d8f-829e-8277e4e1332f" path="/var/lib/kubelet/pods/e9cea652-a164-4d8f-829e-8277e4e1332f/volumes"
Sep 11 01:04:51.112937 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.112916    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"]
Sep 11 01:04:51.113381 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.113359    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9cea652-a164-4d8f-829e-8277e4e1332f" containerName="istio-proxy"
Sep 11 01:04:51.113484 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.113423    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9cea652-a164-4d8f-829e-8277e4e1332f" containerName="istio-proxy"
Sep 11 01:04:51.113591 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.113577    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9cea652-a164-4d8f-829e-8277e4e1332f" containerName="istio-proxy"
Sep 11 01:04:51.124139 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.124113    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:04:51.129666 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.129641    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 01:04:51.130137 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.129926    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-gch5f\""
Sep 11 01:04:51.136511 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.134144    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"]
Sep 11 01:04:51.209104 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.209054    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d5a1311-f9b5-4cb9-8728-20aca33506d9-cert\") pod \"kserve-controller-manager-d6f4b65b8-vxhs5\" (UID: \"9d5a1311-f9b5-4cb9-8728-20aca33506d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:04:51.209340 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.209173    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf64t\" (UniqueName: \"kubernetes.io/projected/9d5a1311-f9b5-4cb9-8728-20aca33506d9-kube-api-access-vf64t\") pod \"kserve-controller-manager-d6f4b65b8-vxhs5\" (UID: \"9d5a1311-f9b5-4cb9-8728-20aca33506d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:04:51.257428 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.257385    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:04:51.257557 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.257465    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:04:51.310159 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.310115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf64t\" (UniqueName: \"kubernetes.io/projected/9d5a1311-f9b5-4cb9-8728-20aca33506d9-kube-api-access-vf64t\") pod \"kserve-controller-manager-d6f4b65b8-vxhs5\" (UID: \"9d5a1311-f9b5-4cb9-8728-20aca33506d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:04:51.310295 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.310252    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d5a1311-f9b5-4cb9-8728-20aca33506d9-cert\") pod \"kserve-controller-manager-d6f4b65b8-vxhs5\" (UID: \"9d5a1311-f9b5-4cb9-8728-20aca33506d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:04:51.310443 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:04:51.310413    2559 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 11 01:04:51.310530 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:04:51.310510    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d5a1311-f9b5-4cb9-8728-20aca33506d9-cert podName:9d5a1311-f9b5-4cb9-8728-20aca33506d9 nodeName:}" failed. No retries permitted until 2026-09-11 01:04:51.81048003 +0000 UTC m=+411.325080197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9d5a1311-f9b5-4cb9-8728-20aca33506d9-cert") pod "kserve-controller-manager-d6f4b65b8-vxhs5" (UID: "9d5a1311-f9b5-4cb9-8728-20aca33506d9") : secret "kserve-webhook-server-cert" not found
Sep 11 01:04:51.336104 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.328219    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf64t\" (UniqueName: \"kubernetes.io/projected/9d5a1311-f9b5-4cb9-8728-20aca33506d9-kube-api-access-vf64t\") pod \"kserve-controller-manager-d6f4b65b8-vxhs5\" (UID: \"9d5a1311-f9b5-4cb9-8728-20aca33506d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:04:51.815287 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.815252    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d5a1311-f9b5-4cb9-8728-20aca33506d9-cert\") pod \"kserve-controller-manager-d6f4b65b8-vxhs5\" (UID: \"9d5a1311-f9b5-4cb9-8728-20aca33506d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:04:51.817776 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:51.817747    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d5a1311-f9b5-4cb9-8728-20aca33506d9-cert\") pod \"kserve-controller-manager-d6f4b65b8-vxhs5\" (UID: \"9d5a1311-f9b5-4cb9-8728-20aca33506d9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:04:52.041051 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:52.041019    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:04:52.136019 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:52.135981    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l" event={"ID":"0b19cad6-8942-43ea-aa5f-01d6bba6f992","Type":"ContainerStarted","Data":"c56570ac6f59e00d73588f4e7ae648cbd83174ab700979af5dedaee54008e778"}
Sep 11 01:04:52.136464 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:52.136070    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:52.137406 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:52.137382    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l"
Sep 11 01:04:52.160023 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:52.159982    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qrb6l" podStartSLOduration=1.61827333 podStartE2EDuration="4.159968579s" totalImagesPullingTime="2.541695249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:52.158819871 +0000 UTC m=+411.673420053" watchObservedRunningTime="2026-09-11 01:04:52.159968579 +0000 UTC m=+411.674568758"
Sep 11 01:04:52.334807 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:04:52.334686    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d5a1311_f9b5_4cb9_8728_20aca33506d9.slice/crio-d3ac51fe3a75793d9b710082c08099825e117edf5fed22aee209f2b56a66a365 WatchSource:0}: Error finding container d3ac51fe3a75793d9b710082c08099825e117edf5fed22aee209f2b56a66a365: Status 404 returned error can't find the container with id d3ac51fe3a75793d9b710082c08099825e117edf5fed22aee209f2b56a66a365
Sep 11 01:04:52.337450 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:52.337429    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"]
Sep 11 01:04:53.142806 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:53.142768    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5" event={"ID":"9d5a1311-f9b5-4cb9-8728-20aca33506d9","Type":"ContainerStarted","Data":"d3ac51fe3a75793d9b710082c08099825e117edf5fed22aee209f2b56a66a365"}
Sep 11 01:04:55.154883 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:55.154847    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5" event={"ID":"9d5a1311-f9b5-4cb9-8728-20aca33506d9","Type":"ContainerStarted","Data":"c1d5ef0c4076ef7b3397ccf71eb71d17cf8a79a537f61054a34ae40262151b82"}
Sep 11 01:04:55.155309 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:55.154917    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:04:55.172650 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:04:55.172590    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5" podStartSLOduration=2.061367923 podStartE2EDuration="4.17257345s" totalImagesPullingTime="2.111205527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:04:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:04:55.17067169 +0000 UTC m=+414.685271875" watchObservedRunningTime="2026-09-11 01:04:55.17257345 +0000 UTC m=+414.687173680"
Sep 11 01:05:26.164761 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:26.164688    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vxhs5"
Sep 11 01:05:48.612432 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.612393    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"]
Sep 11 01:05:48.617388 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.617370    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:48.620054 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.620029    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 01:05:48.620054 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.620053    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 01:05:48.621187 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.621164    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w87xq\""
Sep 11 01:05:48.625262 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.625241    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"]
Sep 11 01:05:48.709081 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.709050    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"]
Sep 11 01:05:48.711039 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.711016    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:48.711113 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.711068    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmmg4\" (UniqueName: \"kubernetes.io/projected/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-kube-api-access-wmmg4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:48.711199 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.711185    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:48.711561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.711547    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:48.720494 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.720472    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"]
Sep 11 01:05:48.809683 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.809652    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"]
Sep 11 01:05:48.811663 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.811636    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:48.811754 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.811698    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmmg4\" (UniqueName: \"kubernetes.io/projected/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-kube-api-access-wmmg4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:48.811793 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.811748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:48.811868 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.811826    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4ldf\" (UniqueName: \"kubernetes.io/projected/510e2ceb-2a7d-4f30-9156-a9a093801307-kube-api-access-z4ldf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:48.811963 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.811941    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:48.812025 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.811996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:48.812231 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.812212    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:48.812320 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.812262    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:48.812591 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.812576    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:48.821719 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.821697    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"]
Sep 11 01:05:48.823235 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.823216    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmmg4\" (UniqueName: \"kubernetes.io/projected/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-kube-api-access-wmmg4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:48.908491 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.908409    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"]
Sep 11 01:05:48.911079 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.911061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:48.917529 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.917495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:48.917675 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.917554    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:48.917675 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.917655    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:48.917801 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.917697    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm65l\" (UniqueName: \"kubernetes.io/projected/f4098a77-3bab-4840-847b-3ff92de9b901-kube-api-access-lm65l\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:48.917801 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.917768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4ldf\" (UniqueName: \"kubernetes.io/projected/510e2ceb-2a7d-4f30-9156-a9a093801307-kube-api-access-z4ldf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:48.917957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.917809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:48.917957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.917924    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:48.918768 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.918745    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:48.921084 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.921062    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"]
Sep 11 01:05:48.926522 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.926500    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4ldf\" (UniqueName: \"kubernetes.io/projected/510e2ceb-2a7d-4f30-9156-a9a093801307-kube-api-access-z4ldf\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:48.927584 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:48.927566    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:49.019124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.019094    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:49.019293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.019154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxfd9\" (UniqueName: \"kubernetes.io/projected/d38222df-211e-40ce-9123-b0f5a823be75-kube-api-access-gxfd9\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:49.019293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.019204    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:49.019293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.019280    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:49.019451 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.019371    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm65l\" (UniqueName: \"kubernetes.io/projected/f4098a77-3bab-4840-847b-3ff92de9b901-kube-api-access-lm65l\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:49.019451 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.019441    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:49.019600 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.019578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:49.019669 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.019621    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:49.020959 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.020941    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:49.027834 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.027810    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm65l\" (UniqueName: \"kubernetes.io/projected/f4098a77-3bab-4840-847b-3ff92de9b901-kube-api-access-lm65l\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:49.120425 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.120401    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:49.120579 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.120451    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:49.120579 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.120501    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxfd9\" (UniqueName: \"kubernetes.io/projected/d38222df-211e-40ce-9123-b0f5a823be75-kube-api-access-gxfd9\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:49.120771 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.120754    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:49.120815 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.120799    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:49.122282 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.122269    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:49.129271 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.129253    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxfd9\" (UniqueName: \"kubernetes.io/projected/d38222df-211e-40ce-9123-b0f5a823be75-kube-api-access-gxfd9\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:49.200793 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.200769    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"]
Sep 11 01:05:49.203179 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:05:49.203150    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb8ecce_4dfd_489c_99c7_28fac7fac98b.slice/crio-2db5affeb2c9f8eb9c0857bb1da245558d7273ad0e98c54f14d08ce45a1cd1be WatchSource:0}: Error finding container 2db5affeb2c9f8eb9c0857bb1da245558d7273ad0e98c54f14d08ce45a1cd1be: Status 404 returned error can't find the container with id 2db5affeb2c9f8eb9c0857bb1da245558d7273ad0e98c54f14d08ce45a1cd1be
Sep 11 01:05:49.226787 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.226764    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:49.291797 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.291775    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"]
Sep 11 01:05:49.294806 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:05:49.294780    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod510e2ceb_2a7d_4f30_9156_a9a093801307.slice/crio-65c789fb5b6b7edd5c336d9bc7956140f6d966b9d872d9803bfee557fca94a3e WatchSource:0}: Error finding container 65c789fb5b6b7edd5c336d9bc7956140f6d966b9d872d9803bfee557fca94a3e: Status 404 returned error can't find the container with id 65c789fb5b6b7edd5c336d9bc7956140f6d966b9d872d9803bfee557fca94a3e
Sep 11 01:05:49.391232 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.391207    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"]
Sep 11 01:05:49.416158 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.415973    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66" event={"ID":"510e2ceb-2a7d-4f30-9156-a9a093801307","Type":"ContainerStarted","Data":"65c789fb5b6b7edd5c336d9bc7956140f6d966b9d872d9803bfee557fca94a3e"}
Sep 11 01:05:49.416760 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:05:49.416715    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4098a77_3bab_4840_847b_3ff92de9b901.slice/crio-f5e2673b4ecff8c41d1583efe91b8a7775c38e118c9e35f9552ee68d406746e2 WatchSource:0}: Error finding container f5e2673b4ecff8c41d1583efe91b8a7775c38e118c9e35f9552ee68d406746e2: Status 404 returned error can't find the container with id f5e2673b4ecff8c41d1583efe91b8a7775c38e118c9e35f9552ee68d406746e2
Sep 11 01:05:49.417820 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.417791    2559 generic.go:356] "Generic (PLEG): container finished" podID="0fb8ecce-4dfd-489c-99c7-28fac7fac98b" containerID="51aa013a2a7b9f2d2ebf6a4dc4959c3764941aa70f1c6cc02b1cc3b9941f5160" exitCode=0
Sep 11 01:05:49.417910 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.417886    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp" event={"ID":"0fb8ecce-4dfd-489c-99c7-28fac7fac98b","Type":"ContainerDied","Data":"51aa013a2a7b9f2d2ebf6a4dc4959c3764941aa70f1c6cc02b1cc3b9941f5160"}
Sep 11 01:05:49.417969 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.417941    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp" event={"ID":"0fb8ecce-4dfd-489c-99c7-28fac7fac98b","Type":"ContainerStarted","Data":"2db5affeb2c9f8eb9c0857bb1da245558d7273ad0e98c54f14d08ce45a1cd1be"}
Sep 11 01:05:49.501217 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:49.501183    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"]
Sep 11 01:05:49.524869 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:05:49.524832    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38222df_211e_40ce_9123_b0f5a823be75.slice/crio-48d0a78b1d3e24c620a28ac25ed3c9ba5350193774db10f04ba6ca2f558e4fe2 WatchSource:0}: Error finding container 48d0a78b1d3e24c620a28ac25ed3c9ba5350193774db10f04ba6ca2f558e4fe2: Status 404 returned error can't find the container with id 48d0a78b1d3e24c620a28ac25ed3c9ba5350193774db10f04ba6ca2f558e4fe2
Sep 11 01:05:50.424573 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:50.424480    2559 generic.go:356] "Generic (PLEG): container finished" podID="510e2ceb-2a7d-4f30-9156-a9a093801307" containerID="bd89e8df1618fe70b54ae95df32316efd860efeff6627e8675948db774bc9d56" exitCode=0
Sep 11 01:05:50.424573 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:50.424534    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66" event={"ID":"510e2ceb-2a7d-4f30-9156-a9a093801307","Type":"ContainerDied","Data":"bd89e8df1618fe70b54ae95df32316efd860efeff6627e8675948db774bc9d56"}
Sep 11 01:05:50.426643 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:50.426624    2559 generic.go:356] "Generic (PLEG): container finished" podID="0fb8ecce-4dfd-489c-99c7-28fac7fac98b" containerID="7b02d35ae1b115de07ce0094d78f364dc9bf9a7aa390d5cd2030d9e894e027bb" exitCode=0
Sep 11 01:05:50.426742 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:50.426692    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp" event={"ID":"0fb8ecce-4dfd-489c-99c7-28fac7fac98b","Type":"ContainerDied","Data":"7b02d35ae1b115de07ce0094d78f364dc9bf9a7aa390d5cd2030d9e894e027bb"}
Sep 11 01:05:50.428413 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:50.428390    2559 generic.go:356] "Generic (PLEG): container finished" podID="f4098a77-3bab-4840-847b-3ff92de9b901" containerID="5772a2b1017946ca8d2937a38a4be916808c5afa3e288ddc8445b54dcd5c6b50" exitCode=0
Sep 11 01:05:50.428574 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:50.428555    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m" event={"ID":"f4098a77-3bab-4840-847b-3ff92de9b901","Type":"ContainerDied","Data":"5772a2b1017946ca8d2937a38a4be916808c5afa3e288ddc8445b54dcd5c6b50"}
Sep 11 01:05:50.428636 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:50.428597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m" event={"ID":"f4098a77-3bab-4840-847b-3ff92de9b901","Type":"ContainerStarted","Data":"f5e2673b4ecff8c41d1583efe91b8a7775c38e118c9e35f9552ee68d406746e2"}
Sep 11 01:05:50.430292 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:50.430272    2559 generic.go:356] "Generic (PLEG): container finished" podID="d38222df-211e-40ce-9123-b0f5a823be75" containerID="39f4ca747f9a7bfb02fcc61bbd0aa7cf8097ecd42341f633cb516af51fb6f687" exitCode=0
Sep 11 01:05:50.430443 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:50.430313    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt" event={"ID":"d38222df-211e-40ce-9123-b0f5a823be75","Type":"ContainerDied","Data":"39f4ca747f9a7bfb02fcc61bbd0aa7cf8097ecd42341f633cb516af51fb6f687"}
Sep 11 01:05:50.430443 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:50.430347    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt" event={"ID":"d38222df-211e-40ce-9123-b0f5a823be75","Type":"ContainerStarted","Data":"48d0a78b1d3e24c620a28ac25ed3c9ba5350193774db10f04ba6ca2f558e4fe2"}
Sep 11 01:05:51.437099 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:51.437069    2559 generic.go:356] "Generic (PLEG): container finished" podID="d38222df-211e-40ce-9123-b0f5a823be75" containerID="d0fc468b0f515910a6eb535aa187b0678582c0a7238d7c0affbcb10109cd25fd" exitCode=0
Sep 11 01:05:51.437413 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:51.437142    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt" event={"ID":"d38222df-211e-40ce-9123-b0f5a823be75","Type":"ContainerDied","Data":"d0fc468b0f515910a6eb535aa187b0678582c0a7238d7c0affbcb10109cd25fd"}
Sep 11 01:05:51.439548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:51.439527    2559 generic.go:356] "Generic (PLEG): container finished" podID="0fb8ecce-4dfd-489c-99c7-28fac7fac98b" containerID="40285abc3b80a65b54ea583cb52e3d8d533d5cfde76de39caf11eb7bab296db0" exitCode=0
Sep 11 01:05:51.439668 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:51.439630    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp" event={"ID":"0fb8ecce-4dfd-489c-99c7-28fac7fac98b","Type":"ContainerDied","Data":"40285abc3b80a65b54ea583cb52e3d8d533d5cfde76de39caf11eb7bab296db0"}
Sep 11 01:05:51.441390 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:51.441370    2559 generic.go:356] "Generic (PLEG): container finished" podID="f4098a77-3bab-4840-847b-3ff92de9b901" containerID="e9f1d282b22ea39876c7ea917dc76e53f3f19cbbd6fda9d6207a7363770c93fa" exitCode=0
Sep 11 01:05:51.441460 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:51.441405    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m" event={"ID":"f4098a77-3bab-4840-847b-3ff92de9b901","Type":"ContainerDied","Data":"e9f1d282b22ea39876c7ea917dc76e53f3f19cbbd6fda9d6207a7363770c93fa"}
Sep 11 01:05:52.448259 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.448227    2559 generic.go:356] "Generic (PLEG): container finished" podID="d38222df-211e-40ce-9123-b0f5a823be75" containerID="ff232a55a06b64aadfc616bd3f6311a4773f69f663e0d5863acef28168937923" exitCode=0
Sep 11 01:05:52.448668 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.448302    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt" event={"ID":"d38222df-211e-40ce-9123-b0f5a823be75","Type":"ContainerDied","Data":"ff232a55a06b64aadfc616bd3f6311a4773f69f663e0d5863acef28168937923"}
Sep 11 01:05:52.450108 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.450086    2559 generic.go:356] "Generic (PLEG): container finished" podID="510e2ceb-2a7d-4f30-9156-a9a093801307" containerID="580fd46244a42c4779b2d2e8cf3130f82270b6691ffe4447e16fd6e98c77a448" exitCode=0
Sep 11 01:05:52.450246 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.450154    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66" event={"ID":"510e2ceb-2a7d-4f30-9156-a9a093801307","Type":"ContainerDied","Data":"580fd46244a42c4779b2d2e8cf3130f82270b6691ffe4447e16fd6e98c77a448"}
Sep 11 01:05:52.452541 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.452520    2559 generic.go:356] "Generic (PLEG): container finished" podID="f4098a77-3bab-4840-847b-3ff92de9b901" containerID="b86e9f01d2b4e9f0f1e981014b399c473567f79855b35e6812f2dd7d3ea4cba3" exitCode=0
Sep 11 01:05:52.452652 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.452599    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m" event={"ID":"f4098a77-3bab-4840-847b-3ff92de9b901","Type":"ContainerDied","Data":"b86e9f01d2b4e9f0f1e981014b399c473567f79855b35e6812f2dd7d3ea4cba3"}
Sep 11 01:05:52.583835 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.583810    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:52.752663 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.752588    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmmg4\" (UniqueName: \"kubernetes.io/projected/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-kube-api-access-wmmg4\") pod \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") "
Sep 11 01:05:52.752812 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.752678    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-bundle\") pod \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") "
Sep 11 01:05:52.752812 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.752699    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-util\") pod \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\" (UID: \"0fb8ecce-4dfd-489c-99c7-28fac7fac98b\") "
Sep 11 01:05:52.753206 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.753183    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-bundle" (OuterVolumeSpecName: "bundle") pod "0fb8ecce-4dfd-489c-99c7-28fac7fac98b" (UID: "0fb8ecce-4dfd-489c-99c7-28fac7fac98b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:05:52.754863 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.754844    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-kube-api-access-wmmg4" (OuterVolumeSpecName: "kube-api-access-wmmg4") pod "0fb8ecce-4dfd-489c-99c7-28fac7fac98b" (UID: "0fb8ecce-4dfd-489c-99c7-28fac7fac98b"). InnerVolumeSpecName "kube-api-access-wmmg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:05:52.758208 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.758189    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-util" (OuterVolumeSpecName: "util") pod "0fb8ecce-4dfd-489c-99c7-28fac7fac98b" (UID: "0fb8ecce-4dfd-489c-99c7-28fac7fac98b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:05:52.853732 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.853646    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmmg4\" (UniqueName: \"kubernetes.io/projected/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-kube-api-access-wmmg4\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:52.853732 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.853676    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:52.853732 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:52.853685    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0fb8ecce-4dfd-489c-99c7-28fac7fac98b-util\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:53.460222 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.460135    2559 generic.go:356] "Generic (PLEG): container finished" podID="510e2ceb-2a7d-4f30-9156-a9a093801307" containerID="6f3c55423b7258bf4c13b2789a83d89abbf9b3300fc85602e8ffd4a0d7736ed0" exitCode=0
Sep 11 01:05:53.460653 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.460218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66" event={"ID":"510e2ceb-2a7d-4f30-9156-a9a093801307","Type":"ContainerDied","Data":"6f3c55423b7258bf4c13b2789a83d89abbf9b3300fc85602e8ffd4a0d7736ed0"}
Sep 11 01:05:53.462219 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.462203    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp"
Sep 11 01:05:53.462310 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.462240    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d78p7gp" event={"ID":"0fb8ecce-4dfd-489c-99c7-28fac7fac98b","Type":"ContainerDied","Data":"2db5affeb2c9f8eb9c0857bb1da245558d7273ad0e98c54f14d08ce45a1cd1be"}
Sep 11 01:05:53.462310 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.462276    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2db5affeb2c9f8eb9c0857bb1da245558d7273ad0e98c54f14d08ce45a1cd1be"
Sep 11 01:05:53.608136 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.608115    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:53.610100 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.610081    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:53.761275 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.761191    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-util\") pod \"d38222df-211e-40ce-9123-b0f5a823be75\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") "
Sep 11 01:05:53.761275 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.761226    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-util\") pod \"f4098a77-3bab-4840-847b-3ff92de9b901\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") "
Sep 11 01:05:53.761275 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.761265    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-bundle\") pod \"f4098a77-3bab-4840-847b-3ff92de9b901\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") "
Sep 11 01:05:53.761563 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.761312    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lm65l\" (UniqueName: \"kubernetes.io/projected/f4098a77-3bab-4840-847b-3ff92de9b901-kube-api-access-lm65l\") pod \"f4098a77-3bab-4840-847b-3ff92de9b901\" (UID: \"f4098a77-3bab-4840-847b-3ff92de9b901\") "
Sep 11 01:05:53.761563 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.761333    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-bundle\") pod \"d38222df-211e-40ce-9123-b0f5a823be75\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") "
Sep 11 01:05:53.761563 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.761364    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxfd9\" (UniqueName: \"kubernetes.io/projected/d38222df-211e-40ce-9123-b0f5a823be75-kube-api-access-gxfd9\") pod \"d38222df-211e-40ce-9123-b0f5a823be75\" (UID: \"d38222df-211e-40ce-9123-b0f5a823be75\") "
Sep 11 01:05:53.761902 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.761873    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-bundle" (OuterVolumeSpecName: "bundle") pod "f4098a77-3bab-4840-847b-3ff92de9b901" (UID: "f4098a77-3bab-4840-847b-3ff92de9b901"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:05:53.762119 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.762093    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-bundle" (OuterVolumeSpecName: "bundle") pod "d38222df-211e-40ce-9123-b0f5a823be75" (UID: "d38222df-211e-40ce-9123-b0f5a823be75"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:05:53.763509 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.763486    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4098a77-3bab-4840-847b-3ff92de9b901-kube-api-access-lm65l" (OuterVolumeSpecName: "kube-api-access-lm65l") pod "f4098a77-3bab-4840-847b-3ff92de9b901" (UID: "f4098a77-3bab-4840-847b-3ff92de9b901"). InnerVolumeSpecName "kube-api-access-lm65l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:05:53.764037 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.764021    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38222df-211e-40ce-9123-b0f5a823be75-kube-api-access-gxfd9" (OuterVolumeSpecName: "kube-api-access-gxfd9") pod "d38222df-211e-40ce-9123-b0f5a823be75" (UID: "d38222df-211e-40ce-9123-b0f5a823be75"). InnerVolumeSpecName "kube-api-access-gxfd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:05:53.766878 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.766853    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-util" (OuterVolumeSpecName: "util") pod "d38222df-211e-40ce-9123-b0f5a823be75" (UID: "d38222df-211e-40ce-9123-b0f5a823be75"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:05:53.767499 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.767481    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-util" (OuterVolumeSpecName: "util") pod "f4098a77-3bab-4840-847b-3ff92de9b901" (UID: "f4098a77-3bab-4840-847b-3ff92de9b901"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:05:53.862374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.862335    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxfd9\" (UniqueName: \"kubernetes.io/projected/d38222df-211e-40ce-9123-b0f5a823be75-kube-api-access-gxfd9\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:53.862374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.862366    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-util\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:53.862374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.862375    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-util\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:53.862374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.862383    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4098a77-3bab-4840-847b-3ff92de9b901-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:53.862657 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.862393    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm65l\" (UniqueName: \"kubernetes.io/projected/f4098a77-3bab-4840-847b-3ff92de9b901-kube-api-access-lm65l\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:53.862657 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:53.862401    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d38222df-211e-40ce-9123-b0f5a823be75-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:54.468857 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.468818    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m"
Sep 11 01:05:54.469239 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.468852    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4crrv8m" event={"ID":"f4098a77-3bab-4840-847b-3ff92de9b901","Type":"ContainerDied","Data":"f5e2673b4ecff8c41d1583efe91b8a7775c38e118c9e35f9552ee68d406746e2"}
Sep 11 01:05:54.469239 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.468906    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5e2673b4ecff8c41d1583efe91b8a7775c38e118c9e35f9552ee68d406746e2"
Sep 11 01:05:54.470884 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.470859    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt"
Sep 11 01:05:54.471014 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.470881    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aq27pt" event={"ID":"d38222df-211e-40ce-9123-b0f5a823be75","Type":"ContainerDied","Data":"48d0a78b1d3e24c620a28ac25ed3c9ba5350193774db10f04ba6ca2f558e4fe2"}
Sep 11 01:05:54.471014 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.470928    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48d0a78b1d3e24c620a28ac25ed3c9ba5350193774db10f04ba6ca2f558e4fe2"
Sep 11 01:05:54.597914 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.597893    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:54.768548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.768517    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-util\") pod \"510e2ceb-2a7d-4f30-9156-a9a093801307\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") "
Sep 11 01:05:54.768734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.768592    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-bundle\") pod \"510e2ceb-2a7d-4f30-9156-a9a093801307\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") "
Sep 11 01:05:54.768734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.768691    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4ldf\" (UniqueName: \"kubernetes.io/projected/510e2ceb-2a7d-4f30-9156-a9a093801307-kube-api-access-z4ldf\") pod \"510e2ceb-2a7d-4f30-9156-a9a093801307\" (UID: \"510e2ceb-2a7d-4f30-9156-a9a093801307\") "
Sep 11 01:05:54.769156 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.769129    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-bundle" (OuterVolumeSpecName: "bundle") pod "510e2ceb-2a7d-4f30-9156-a9a093801307" (UID: "510e2ceb-2a7d-4f30-9156-a9a093801307"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:05:54.770777 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.770755    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/510e2ceb-2a7d-4f30-9156-a9a093801307-kube-api-access-z4ldf" (OuterVolumeSpecName: "kube-api-access-z4ldf") pod "510e2ceb-2a7d-4f30-9156-a9a093801307" (UID: "510e2ceb-2a7d-4f30-9156-a9a093801307"). InnerVolumeSpecName "kube-api-access-z4ldf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:05:54.776189 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.776165    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-util" (OuterVolumeSpecName: "util") pod "510e2ceb-2a7d-4f30-9156-a9a093801307" (UID: "510e2ceb-2a7d-4f30-9156-a9a093801307"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:05:54.870242 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.870163    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-util\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:54.870242 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.870189    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/510e2ceb-2a7d-4f30-9156-a9a093801307-bundle\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:54.870242 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:54.870199    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4ldf\" (UniqueName: \"kubernetes.io/projected/510e2ceb-2a7d-4f30-9156-a9a093801307-kube-api-access-z4ldf\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:05:55.479575 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:55.479545    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66"
Sep 11 01:05:55.479952 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:55.479581    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c685h66" event={"ID":"510e2ceb-2a7d-4f30-9156-a9a093801307","Type":"ContainerDied","Data":"65c789fb5b6b7edd5c336d9bc7956140f6d966b9d872d9803bfee557fca94a3e"}
Sep 11 01:05:55.479952 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:05:55.479640    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65c789fb5b6b7edd5c336d9bc7956140f6d966b9d872d9803bfee557fca94a3e"
Sep 11 01:06:00.641338 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641301    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"]
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641660    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="510e2ceb-2a7d-4f30-9156-a9a093801307" containerName="util"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641672    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="510e2ceb-2a7d-4f30-9156-a9a093801307" containerName="util"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641680    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4098a77-3bab-4840-847b-3ff92de9b901" containerName="pull"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641685    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4098a77-3bab-4840-847b-3ff92de9b901" containerName="pull"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641692    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fb8ecce-4dfd-489c-99c7-28fac7fac98b" containerName="pull"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641697    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fb8ecce-4dfd-489c-99c7-28fac7fac98b" containerName="pull"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641709    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4098a77-3bab-4840-847b-3ff92de9b901" containerName="util"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641714    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4098a77-3bab-4840-847b-3ff92de9b901" containerName="util"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641719    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38222df-211e-40ce-9123-b0f5a823be75" containerName="util"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641724    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38222df-211e-40ce-9123-b0f5a823be75" containerName="util"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641731    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38222df-211e-40ce-9123-b0f5a823be75" containerName="pull"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641736    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38222df-211e-40ce-9123-b0f5a823be75" containerName="pull"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641742    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="510e2ceb-2a7d-4f30-9156-a9a093801307" containerName="extract"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641748    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="510e2ceb-2a7d-4f30-9156-a9a093801307" containerName="extract"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641764    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fb8ecce-4dfd-489c-99c7-28fac7fac98b" containerName="extract"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641769    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fb8ecce-4dfd-489c-99c7-28fac7fac98b" containerName="extract"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641777    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38222df-211e-40ce-9123-b0f5a823be75" containerName="extract"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641785    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38222df-211e-40ce-9123-b0f5a823be75" containerName="extract"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641798    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="510e2ceb-2a7d-4f30-9156-a9a093801307" containerName="pull"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641805    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="510e2ceb-2a7d-4f30-9156-a9a093801307" containerName="pull"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641816    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4098a77-3bab-4840-847b-3ff92de9b901" containerName="extract"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641824    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4098a77-3bab-4840-847b-3ff92de9b901" containerName="extract"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641845    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fb8ecce-4dfd-489c-99c7-28fac7fac98b" containerName="util"
Sep 11 01:06:00.641887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641852    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fb8ecce-4dfd-489c-99c7-28fac7fac98b" containerName="util"
Sep 11 01:06:00.642706 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641932    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="510e2ceb-2a7d-4f30-9156-a9a093801307" containerName="extract"
Sep 11 01:06:00.642706 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641942    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4098a77-3bab-4840-847b-3ff92de9b901" containerName="extract"
Sep 11 01:06:00.642706 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641950    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fb8ecce-4dfd-489c-99c7-28fac7fac98b" containerName="extract"
Sep 11 01:06:00.642706 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.641956    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38222df-211e-40ce-9123-b0f5a823be75" containerName="extract"
Sep 11 01:06:00.645104 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.645088    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:00.648615 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.648580    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 01:06:00.649666 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.649648    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gkmpg\""
Sep 11 01:06:00.649763 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.649686    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 01:06:00.657367 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.657341    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"]
Sep 11 01:06:00.720332 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.720297    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmm78\" (UniqueName: \"kubernetes.io/projected/dc5a5eff-ab50-4e3a-8de4-19e4606da1c7-kube-api-access-mmm78\") pod \"limitador-operator-controller-manager-74495c69f8-mm4j6\" (UID: \"dc5a5eff-ab50-4e3a-8de4-19e4606da1c7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:00.821393 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.821362    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmm78\" (UniqueName: \"kubernetes.io/projected/dc5a5eff-ab50-4e3a-8de4-19e4606da1c7-kube-api-access-mmm78\") pod \"limitador-operator-controller-manager-74495c69f8-mm4j6\" (UID: \"dc5a5eff-ab50-4e3a-8de4-19e4606da1c7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:00.829998 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.829967    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmm78\" (UniqueName: \"kubernetes.io/projected/dc5a5eff-ab50-4e3a-8de4-19e4606da1c7-kube-api-access-mmm78\") pod \"limitador-operator-controller-manager-74495c69f8-mm4j6\" (UID: \"dc5a5eff-ab50-4e3a-8de4-19e4606da1c7\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:00.959180 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.959102    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gkmpg\""
Sep 11 01:06:00.967425 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:00.967399    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:01.247721 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:01.247694    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"]
Sep 11 01:06:01.249064 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:06:01.249039    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5a5eff_ab50_4e3a_8de4_19e4606da1c7.slice/crio-f53ed536b2ba660c14e3f805a14869b04d3806c706c4f6311d1fd347ebad836e WatchSource:0}: Error finding container f53ed536b2ba660c14e3f805a14869b04d3806c706c4f6311d1fd347ebad836e: Status 404 returned error can't find the container with id f53ed536b2ba660c14e3f805a14869b04d3806c706c4f6311d1fd347ebad836e
Sep 11 01:06:01.516652 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:01.516621    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6" event={"ID":"dc5a5eff-ab50-4e3a-8de4-19e4606da1c7","Type":"ContainerStarted","Data":"f53ed536b2ba660c14e3f805a14869b04d3806c706c4f6311d1fd347ebad836e"}
Sep 11 01:06:03.079973 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:03.079930    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-xb4vc"]
Sep 11 01:06:03.096268 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:03.096231    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-xb4vc"]
Sep 11 01:06:03.096464 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:03.096367    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-xb4vc"
Sep 11 01:06:03.099348 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:03.099323    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-mvmfv\""
Sep 11 01:06:03.138592 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:03.138551    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lfz9\" (UniqueName: \"kubernetes.io/projected/6695ef9d-66d4-4bd2-90c5-f62f77948ae8-kube-api-access-8lfz9\") pod \"authorino-operator-76b478bd78-xb4vc\" (UID: \"6695ef9d-66d4-4bd2-90c5-f62f77948ae8\") " pod="kuadrant-system/authorino-operator-76b478bd78-xb4vc"
Sep 11 01:06:03.240398 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:03.239987    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lfz9\" (UniqueName: \"kubernetes.io/projected/6695ef9d-66d4-4bd2-90c5-f62f77948ae8-kube-api-access-8lfz9\") pod \"authorino-operator-76b478bd78-xb4vc\" (UID: \"6695ef9d-66d4-4bd2-90c5-f62f77948ae8\") " pod="kuadrant-system/authorino-operator-76b478bd78-xb4vc"
Sep 11 01:06:03.254002 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:03.253973    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lfz9\" (UniqueName: \"kubernetes.io/projected/6695ef9d-66d4-4bd2-90c5-f62f77948ae8-kube-api-access-8lfz9\") pod \"authorino-operator-76b478bd78-xb4vc\" (UID: \"6695ef9d-66d4-4bd2-90c5-f62f77948ae8\") " pod="kuadrant-system/authorino-operator-76b478bd78-xb4vc"
Sep 11 01:06:03.424195 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:03.424114    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-xb4vc"
Sep 11 01:06:04.000829 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:04.000804    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-xb4vc"]
Sep 11 01:06:04.004922 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:06:04.004888    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6695ef9d_66d4_4bd2_90c5_f62f77948ae8.slice/crio-ab56c84e27da9804e04250727de2aebac5b7b1ba321171b9fd7d3ba997a980a2 WatchSource:0}: Error finding container ab56c84e27da9804e04250727de2aebac5b7b1ba321171b9fd7d3ba997a980a2: Status 404 returned error can't find the container with id ab56c84e27da9804e04250727de2aebac5b7b1ba321171b9fd7d3ba997a980a2
Sep 11 01:06:04.536563 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:04.536528    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-xb4vc" event={"ID":"6695ef9d-66d4-4bd2-90c5-f62f77948ae8","Type":"ContainerStarted","Data":"ab56c84e27da9804e04250727de2aebac5b7b1ba321171b9fd7d3ba997a980a2"}
Sep 11 01:06:04.538400 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:04.538370    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6" event={"ID":"dc5a5eff-ab50-4e3a-8de4-19e4606da1c7","Type":"ContainerStarted","Data":"c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e"}
Sep 11 01:06:04.538522 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:04.538462    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:04.561959 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:04.561897    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6" podStartSLOduration=2.018447577 podStartE2EDuration="4.561875003s" totalImagesPullingTime="2.543427426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:06:04.558294199 +0000 UTC m=+484.072894399" watchObservedRunningTime="2026-09-11 01:06:04.561875003 +0000 UTC m=+484.076475202"
Sep 11 01:06:06.551066 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:06.551027    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-xb4vc" event={"ID":"6695ef9d-66d4-4bd2-90c5-f62f77948ae8","Type":"ContainerStarted","Data":"ac7f73a1aae357f8f7b677082dbe44fa3502648e34a1374267695aaa4bd86d33"}
Sep 11 01:06:06.551525 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:06.551163    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-xb4vc"
Sep 11 01:06:06.575963 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:06.575916    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-xb4vc" podStartSLOduration=2.054410584 podStartE2EDuration="3.57590177s" totalImagesPullingTime="1.521491186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:06:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:06:06.573807038 +0000 UTC m=+486.088407221" watchObservedRunningTime="2026-09-11 01:06:06.57590177 +0000 UTC m=+486.090501961"
Sep 11 01:06:15.546155 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:15.546127    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:17.558165 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:17.558129    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-xb4vc"
Sep 11 01:06:25.646437 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:25.646400    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"]
Sep 11 01:06:25.646923 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:25.646592    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6" podUID="dc5a5eff-ab50-4e3a-8de4-19e4606da1c7" containerName="manager" containerID="cri-o://c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e" gracePeriod=2
Sep 11 01:06:25.665939 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:25.665910    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"]
Sep 11 01:06:25.673570 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:06:25.673528    2559 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"dc5a5eff-ab50-4e3a-8de4-19e4606da1c7\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-mm4j6\": pods \"limitador-operator-controller-manager-74495c69f8-mm4j6\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:25.889792 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:25.889756    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:25.892085 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:06:25.892061    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mm4j6\" is forbidden: User \"system:node:ip-10-0-138-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-14.ec2.internal' and this object" podUID="dc5a5eff-ab50-4e3a-8de4-19e4606da1c7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:25.942361 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:25.942274    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmm78\" (UniqueName: \"kubernetes.io/projected/dc5a5eff-ab50-4e3a-8de4-19e4606da1c7-kube-api-access-mmm78\") pod \"dc5a5eff-ab50-4e3a-8de4-19e4606da1c7\" (UID: \"dc5a5eff-ab50-4e3a-8de4-19e4606da1c7\") "
Sep 11 01:06:25.944457 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:25.944427    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc5a5eff-ab50-4e3a-8de4-19e4606da1c7-kube-api-access-mmm78" (OuterVolumeSpecName: "kube-api-access-mmm78") pod "dc5a5eff-ab50-4e3a-8de4-19e4606da1c7" (UID: "dc5a5eff-ab50-4e3a-8de4-19e4606da1c7"). InnerVolumeSpecName "kube-api-access-mmm78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:06:26.043014 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:26.042981    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmm78\" (UniqueName: \"kubernetes.io/projected/dc5a5eff-ab50-4e3a-8de4-19e4606da1c7-kube-api-access-mmm78\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:06:26.665505 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:26.665470    2559 generic.go:356] "Generic (PLEG): container finished" podID="dc5a5eff-ab50-4e3a-8de4-19e4606da1c7" containerID="c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e" exitCode=0
Sep 11 01:06:26.665988 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:26.665512    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:26.665988 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:26.665523    2559 scope.go:117] "RemoveContainer" containerID="c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e"
Sep 11 01:06:26.668060 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:06:26.668035    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mm4j6\" is forbidden: User \"system:node:ip-10-0-138-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-14.ec2.internal' and this object" podUID="dc5a5eff-ab50-4e3a-8de4-19e4606da1c7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:26.675165 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:26.675141    2559 scope.go:117] "RemoveContainer" containerID="c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e"
Sep 11 01:06:26.675433 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:06:26.675415    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e\": container with ID starting with c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e not found: ID does not exist" containerID="c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e"
Sep 11 01:06:26.675482 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:26.675442    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e"} err="failed to get container status \"c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e\": rpc error: code = NotFound desc = could not find container \"c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e\": container with ID starting with c54b60f632164cd6c03f813c6e403b1b59e1d6ceaddb9a544798b89cef0bc70e not found: ID does not exist"
Sep 11 01:06:26.676326 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:06:26.676303    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mm4j6\" is forbidden: User \"system:node:ip-10-0-138-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-14.ec2.internal' and this object" podUID="dc5a5eff-ab50-4e3a-8de4-19e4606da1c7" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mm4j6"
Sep 11 01:06:27.116093 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:27.116060    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc5a5eff-ab50-4e3a-8de4-19e4606da1c7" path="/var/lib/kubelet/pods/dc5a5eff-ab50-4e3a-8de4-19e4606da1c7/volumes"
Sep 11 01:06:53.945825 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:53.945748    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"]
Sep 11 01:06:53.946204 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:53.946169    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc5a5eff-ab50-4e3a-8de4-19e4606da1c7" containerName="manager"
Sep 11 01:06:53.946204 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:53.946182    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc5a5eff-ab50-4e3a-8de4-19e4606da1c7" containerName="manager"
Sep 11 01:06:53.946289 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:53.946271    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc5a5eff-ab50-4e3a-8de4-19e4606da1c7" containerName="manager"
Sep 11 01:06:53.950903 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:53.950885    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:53.953454 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:53.953431    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zjt5z\""
Sep 11 01:06:53.959854 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:53.959833    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"]
Sep 11 01:06:54.094030 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.093989    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.094233 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.094038    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be6b052f-904e-4764-950e-fb94ac2d1571-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.094233 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.094070    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be6b052f-904e-4764-950e-fb94ac2d1571-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.094233 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.094132    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tjhc\" (UniqueName: \"kubernetes.io/projected/be6b052f-904e-4764-950e-fb94ac2d1571-kube-api-access-6tjhc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.094233 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.094170    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.094233 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.094210    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be6b052f-904e-4764-950e-fb94ac2d1571-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.094469 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.094248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.094469 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.094284    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.094469 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.094322    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.195509 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.195475    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tjhc\" (UniqueName: \"kubernetes.io/projected/be6b052f-904e-4764-950e-fb94ac2d1571-kube-api-access-6tjhc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.195797 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.195775    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.196013 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.195958    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be6b052f-904e-4764-950e-fb94ac2d1571-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.196188 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.196146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.196305 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.196201    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.196305 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.196242    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.196305 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.196291    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.196480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.196321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be6b052f-904e-4764-950e-fb94ac2d1571-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.196480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.196359    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be6b052f-904e-4764-950e-fb94ac2d1571-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.196706 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.196668    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.196791 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.196736    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.196915 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.196893    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be6b052f-904e-4764-950e-fb94ac2d1571-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.197022 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.197000    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.197330 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.197310    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.198922 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.198904    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be6b052f-904e-4764-950e-fb94ac2d1571-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.199300 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.199284    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be6b052f-904e-4764-950e-fb94ac2d1571-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.203213 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.203183    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tjhc\" (UniqueName: \"kubernetes.io/projected/be6b052f-904e-4764-950e-fb94ac2d1571-kube-api-access-6tjhc\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.203853 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.203832    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be6b052f-904e-4764-950e-fb94ac2d1571-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f72st\" (UID: \"be6b052f-904e-4764-950e-fb94ac2d1571\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.261672 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.261634    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:54.536870 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.536831    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"]
Sep 11 01:06:54.538023 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:06:54.537998    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe6b052f_904e_4764_950e_fb94ac2d1571.slice/crio-51845ca56a17483bcf4761ced0627d6b48c366377b6351520db054cb8c448c5d WatchSource:0}: Error finding container 51845ca56a17483bcf4761ced0627d6b48c366377b6351520db054cb8c448c5d: Status 404 returned error can't find the container with id 51845ca56a17483bcf4761ced0627d6b48c366377b6351520db054cb8c448c5d
Sep 11 01:06:54.540160 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.540122    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:06:54.540245 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.540188    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:06:54.540245 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.540218    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:06:54.813791 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.813707    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st" event={"ID":"be6b052f-904e-4764-950e-fb94ac2d1571","Type":"ContainerStarted","Data":"dc8d29db32170c75c04bc68678f55701f5fadcd9e137181c0207fc9508567594"}
Sep 11 01:06:54.813791 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.813782    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st" event={"ID":"be6b052f-904e-4764-950e-fb94ac2d1571","Type":"ContainerStarted","Data":"51845ca56a17483bcf4761ced0627d6b48c366377b6351520db054cb8c448c5d"}
Sep 11 01:06:54.836664 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:54.836593    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st" podStartSLOduration=1.836578214 podStartE2EDuration="1.836578214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:06:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:06:54.833145101 +0000 UTC m=+534.347745283" watchObservedRunningTime="2026-09-11 01:06:54.836578214 +0000 UTC m=+534.351178407"
Sep 11 01:06:55.262510 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:55.262420    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:55.266973 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:55.266946    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:55.819421 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:55.819395    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:06:55.820482 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:06:55.820464    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:07:08.882615 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:08.882579    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hw52p"]
Sep 11 01:07:08.885599 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:08.885584    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:08.888021 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:08.887997    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 01:07:08.888128 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:08.888045    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7d58x\""
Sep 11 01:07:08.895010 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:08.894987    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hw52p"]
Sep 11 01:07:08.926402 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:08.926370    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f6f295d4-a668-466d-9cd5-707b9f3386f1-config-file\") pod \"limitador-limitador-8588749977-hw52p\" (UID: \"f6f295d4-a668-466d-9cd5-707b9f3386f1\") " pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:08.926559 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:08.926424    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh7wr\" (UniqueName: \"kubernetes.io/projected/f6f295d4-a668-466d-9cd5-707b9f3386f1-kube-api-access-vh7wr\") pod \"limitador-limitador-8588749977-hw52p\" (UID: \"f6f295d4-a668-466d-9cd5-707b9f3386f1\") " pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:08.979853 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:08.979815    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hw52p"]
Sep 11 01:07:09.027362 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:09.027332    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f6f295d4-a668-466d-9cd5-707b9f3386f1-config-file\") pod \"limitador-limitador-8588749977-hw52p\" (UID: \"f6f295d4-a668-466d-9cd5-707b9f3386f1\") " pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:09.027518 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:09.027394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh7wr\" (UniqueName: \"kubernetes.io/projected/f6f295d4-a668-466d-9cd5-707b9f3386f1-kube-api-access-vh7wr\") pod \"limitador-limitador-8588749977-hw52p\" (UID: \"f6f295d4-a668-466d-9cd5-707b9f3386f1\") " pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:09.027989 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:09.027969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f6f295d4-a668-466d-9cd5-707b9f3386f1-config-file\") pod \"limitador-limitador-8588749977-hw52p\" (UID: \"f6f295d4-a668-466d-9cd5-707b9f3386f1\") " pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:09.035476 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:09.035452    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh7wr\" (UniqueName: \"kubernetes.io/projected/f6f295d4-a668-466d-9cd5-707b9f3386f1-kube-api-access-vh7wr\") pod \"limitador-limitador-8588749977-hw52p\" (UID: \"f6f295d4-a668-466d-9cd5-707b9f3386f1\") " pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:09.197827 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:09.197751    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:09.481170 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:09.481092    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hw52p"]
Sep 11 01:07:09.484530 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:07:09.484502    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f295d4_a668_466d_9cd5_707b9f3386f1.slice/crio-2738d275eb85130b7eddb9c3a03875fb2c47eaab256aa1b19db0417ed770685e WatchSource:0}: Error finding container 2738d275eb85130b7eddb9c3a03875fb2c47eaab256aa1b19db0417ed770685e: Status 404 returned error can't find the container with id 2738d275eb85130b7eddb9c3a03875fb2c47eaab256aa1b19db0417ed770685e
Sep 11 01:07:09.898599 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:09.898567    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hw52p" event={"ID":"f6f295d4-a668-466d-9cd5-707b9f3386f1","Type":"ContainerStarted","Data":"2738d275eb85130b7eddb9c3a03875fb2c47eaab256aa1b19db0417ed770685e"}
Sep 11 01:07:10.905210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:10.905173    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hw52p" event={"ID":"f6f295d4-a668-466d-9cd5-707b9f3386f1","Type":"ContainerStarted","Data":"867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593"}
Sep 11 01:07:10.905740 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:10.905372    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:10.923094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:10.923046    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-hw52p" podStartSLOduration=1.961019101 podStartE2EDuration="2.923027409s" totalImagesPullingTime="962.008308ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:07:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:07:10.922156661 +0000 UTC m=+550.436756846" watchObservedRunningTime="2026-09-11 01:07:10.923027409 +0000 UTC m=+550.437627592"
Sep 11 01:07:21.911157 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:21.911127    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:22.745024 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:22.744987    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hw52p"]
Sep 11 01:07:22.745210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:22.745164    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-hw52p" podUID="f6f295d4-a668-466d-9cd5-707b9f3386f1" containerName="limitador" containerID="cri-o://867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593" gracePeriod=30
Sep 11 01:07:23.296243 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.296221    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:23.461517 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.461424    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f6f295d4-a668-466d-9cd5-707b9f3386f1-config-file\") pod \"f6f295d4-a668-466d-9cd5-707b9f3386f1\" (UID: \"f6f295d4-a668-466d-9cd5-707b9f3386f1\") "
Sep 11 01:07:23.461723 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.461535    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh7wr\" (UniqueName: \"kubernetes.io/projected/f6f295d4-a668-466d-9cd5-707b9f3386f1-kube-api-access-vh7wr\") pod \"f6f295d4-a668-466d-9cd5-707b9f3386f1\" (UID: \"f6f295d4-a668-466d-9cd5-707b9f3386f1\") "
Sep 11 01:07:23.461820 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.461791    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6f295d4-a668-466d-9cd5-707b9f3386f1-config-file" (OuterVolumeSpecName: "config-file") pod "f6f295d4-a668-466d-9cd5-707b9f3386f1" (UID: "f6f295d4-a668-466d-9cd5-707b9f3386f1"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:07:23.463650 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.463626    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6f295d4-a668-466d-9cd5-707b9f3386f1-kube-api-access-vh7wr" (OuterVolumeSpecName: "kube-api-access-vh7wr") pod "f6f295d4-a668-466d-9cd5-707b9f3386f1" (UID: "f6f295d4-a668-466d-9cd5-707b9f3386f1"). InnerVolumeSpecName "kube-api-access-vh7wr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:07:23.563106 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.563069    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh7wr\" (UniqueName: \"kubernetes.io/projected/f6f295d4-a668-466d-9cd5-707b9f3386f1-kube-api-access-vh7wr\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:07:23.563106 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.563100    2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f6f295d4-a668-466d-9cd5-707b9f3386f1-config-file\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:07:23.973537 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.973503    2559 generic.go:356] "Generic (PLEG): container finished" podID="f6f295d4-a668-466d-9cd5-707b9f3386f1" containerID="867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593" exitCode=0
Sep 11 01:07:23.973725 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.973557    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hw52p"
Sep 11 01:07:23.973725 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.973583    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hw52p" event={"ID":"f6f295d4-a668-466d-9cd5-707b9f3386f1","Type":"ContainerDied","Data":"867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593"}
Sep 11 01:07:23.973725 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.973681    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hw52p" event={"ID":"f6f295d4-a668-466d-9cd5-707b9f3386f1","Type":"ContainerDied","Data":"2738d275eb85130b7eddb9c3a03875fb2c47eaab256aa1b19db0417ed770685e"}
Sep 11 01:07:23.973725 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.973710    2559 scope.go:117] "RemoveContainer" containerID="867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593"
Sep 11 01:07:23.982562 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.982544    2559 scope.go:117] "RemoveContainer" containerID="867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593"
Sep 11 01:07:23.982902 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:07:23.982884    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593\": container with ID starting with 867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593 not found: ID does not exist" containerID="867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593"
Sep 11 01:07:23.982962 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.982911    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593"} err="failed to get container status \"867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593\": rpc error: code = NotFound desc = could not find container \"867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593\": container with ID starting with 867ade72ae549d83b17c146b449b3c6072885cc0cc8399bc08fcd1825635e593 not found: ID does not exist"
Sep 11 01:07:23.998008 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:23.997975    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hw52p"]
Sep 11 01:07:24.000136 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:24.000114    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hw52p"]
Sep 11 01:07:25.109522 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:25.109486    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6f295d4-a668-466d-9cd5-707b9f3386f1" path="/var/lib/kubelet/pods/f6f295d4-a668-466d-9cd5-707b9f3386f1/volumes"
Sep 11 01:07:27.986354 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:27.986320    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"]
Sep 11 01:07:27.986760 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:27.986706    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6f295d4-a668-466d-9cd5-707b9f3386f1" containerName="limitador"
Sep 11 01:07:27.986760 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:27.986722    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6f295d4-a668-466d-9cd5-707b9f3386f1" containerName="limitador"
Sep 11 01:07:27.986854 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:27.986790    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6f295d4-a668-466d-9cd5-707b9f3386f1" containerName="limitador"
Sep 11 01:07:27.988925 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:27.988909    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"
Sep 11 01:07:27.992259 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:27.992239    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 01:07:27.992390 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:27.992292    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-xlj9v\""
Sep 11 01:07:27.992390 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:27.992361    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 01:07:27.993277 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:27.993259    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 01:07:28.001798 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:28.001769    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"]
Sep 11 01:07:28.101833 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:28.101800    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh4dd\" (UniqueName: \"kubernetes.io/projected/9e30efd1-a473-46f9-835f-2e458a737970-kube-api-access-sh4dd\") pod \"postgres-69fb4594cf-pnvcw\" (UID: \"9e30efd1-a473-46f9-835f-2e458a737970\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"
Sep 11 01:07:28.101983 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:28.101857    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9e30efd1-a473-46f9-835f-2e458a737970-data\") pod \"postgres-69fb4594cf-pnvcw\" (UID: \"9e30efd1-a473-46f9-835f-2e458a737970\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"
Sep 11 01:07:28.202907 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:28.202872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9e30efd1-a473-46f9-835f-2e458a737970-data\") pod \"postgres-69fb4594cf-pnvcw\" (UID: \"9e30efd1-a473-46f9-835f-2e458a737970\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"
Sep 11 01:07:28.203062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:28.202979    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh4dd\" (UniqueName: \"kubernetes.io/projected/9e30efd1-a473-46f9-835f-2e458a737970-kube-api-access-sh4dd\") pod \"postgres-69fb4594cf-pnvcw\" (UID: \"9e30efd1-a473-46f9-835f-2e458a737970\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"
Sep 11 01:07:28.203271 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:28.203251    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9e30efd1-a473-46f9-835f-2e458a737970-data\") pod \"postgres-69fb4594cf-pnvcw\" (UID: \"9e30efd1-a473-46f9-835f-2e458a737970\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"
Sep 11 01:07:28.211980 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:28.211956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh4dd\" (UniqueName: \"kubernetes.io/projected/9e30efd1-a473-46f9-835f-2e458a737970-kube-api-access-sh4dd\") pod \"postgres-69fb4594cf-pnvcw\" (UID: \"9e30efd1-a473-46f9-835f-2e458a737970\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"
Sep 11 01:07:28.300341 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:28.300298    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"
Sep 11 01:07:28.568451 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:28.568424    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"]
Sep 11 01:07:28.569978 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:07:28.569944    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e30efd1_a473_46f9_835f_2e458a737970.slice/crio-996edb9b65cb863f3cd811c6f27342d3c3a5777b0a5c403f27fd93966e5166ff WatchSource:0}: Error finding container 996edb9b65cb863f3cd811c6f27342d3c3a5777b0a5c403f27fd93966e5166ff: Status 404 returned error can't find the container with id 996edb9b65cb863f3cd811c6f27342d3c3a5777b0a5c403f27fd93966e5166ff
Sep 11 01:07:29.004457 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:29.004370    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw" event={"ID":"9e30efd1-a473-46f9-835f-2e458a737970","Type":"ContainerStarted","Data":"996edb9b65cb863f3cd811c6f27342d3c3a5777b0a5c403f27fd93966e5166ff"}
Sep 11 01:07:34.036127 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:34.036090    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw" event={"ID":"9e30efd1-a473-46f9-835f-2e458a737970","Type":"ContainerStarted","Data":"f3dfebe32d907b7902a755a1ca7583424150e6da91ee8e1fcc0494997afc5b2f"}
Sep 11 01:07:34.036505 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:34.036216    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"
Sep 11 01:07:34.051266 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:34.051221    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw" podStartSLOduration=2.475423434 podStartE2EDuration="7.051205198s" totalImagesPullingTime="4.575781764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:07:34.050062056 +0000 UTC m=+573.564662239" watchObservedRunningTime="2026-09-11 01:07:34.051205198 +0000 UTC m=+573.565805436"
Sep 11 01:07:40.071204 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:40.071173    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pnvcw"
Sep 11 01:07:47.880387 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:47.880352    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55d86fb7c8-jjf26"]
Sep 11 01:07:47.899006 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:47.898953    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55d86fb7c8-jjf26"]
Sep 11 01:07:47.899185 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:47.899094    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:47.901753 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:47.901729    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 01:07:47.902046 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:47.901729    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 01:07:47.902148 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:47.901748    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 01:07:47.902220 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:47.902024    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-r4szl\""
Sep 11 01:07:47.970371 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:47.970337    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lscj2\" (UniqueName: \"kubernetes.io/projected/c8b21b39-6bcb-497c-ae07-953e32e45541-kube-api-access-lscj2\") pod \"maas-controller-55d86fb7c8-jjf26\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") " pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:47.970550 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:47.970441    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-metrics-certs\") pod \"maas-controller-55d86fb7c8-jjf26\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") " pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:47.970550 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:47.970478    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-webhook-cert\") pod \"maas-controller-55d86fb7c8-jjf26\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") " pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:48.071161 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.071127    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-metrics-certs\") pod \"maas-controller-55d86fb7c8-jjf26\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") " pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:48.071332 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.071172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-webhook-cert\") pod \"maas-controller-55d86fb7c8-jjf26\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") " pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:48.071332 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.071220    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lscj2\" (UniqueName: \"kubernetes.io/projected/c8b21b39-6bcb-497c-ae07-953e32e45541-kube-api-access-lscj2\") pod \"maas-controller-55d86fb7c8-jjf26\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") " pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:48.071332 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:07:48.071274    2559 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 11 01:07:48.071449 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:07:48.071357    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-metrics-certs podName:c8b21b39-6bcb-497c-ae07-953e32e45541 nodeName:}" failed. No retries permitted until 2026-09-11 01:07:48.571334023 +0000 UTC m=+588.085934196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-metrics-certs") pod "maas-controller-55d86fb7c8-jjf26" (UID: "c8b21b39-6bcb-497c-ae07-953e32e45541") : secret "maas-controller-metrics-tls" not found
Sep 11 01:07:48.074115 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.074090    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-webhook-cert\") pod \"maas-controller-55d86fb7c8-jjf26\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") " pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:48.074561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.074542    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-kc22x"]
Sep 11 01:07:48.081734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.081714    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.082670 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.082645    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lscj2\" (UniqueName: \"kubernetes.io/projected/c8b21b39-6bcb-497c-ae07-953e32e45541-kube-api-access-lscj2\") pod \"maas-controller-55d86fb7c8-jjf26\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") " pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:48.090278 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.090255    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-kc22x"]
Sep 11 01:07:48.171965 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.171877    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-webhook-cert\") pod \"maas-controller-6c6d6b6d8d-kc22x\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") " pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.171965 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.171941    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-metrics-certs\") pod \"maas-controller-6c6d6b6d8d-kc22x\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") " pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.172163 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.172012    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l6h7\" (UniqueName: \"kubernetes.io/projected/486bee5f-d3b7-4133-a03a-4f75a4736f7d-kube-api-access-4l6h7\") pod \"maas-controller-6c6d6b6d8d-kc22x\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") " pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.273110 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.273066    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-metrics-certs\") pod \"maas-controller-6c6d6b6d8d-kc22x\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") " pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.273278 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.273152    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l6h7\" (UniqueName: \"kubernetes.io/projected/486bee5f-d3b7-4133-a03a-4f75a4736f7d-kube-api-access-4l6h7\") pod \"maas-controller-6c6d6b6d8d-kc22x\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") " pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.273278 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.273198    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-webhook-cert\") pod \"maas-controller-6c6d6b6d8d-kc22x\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") " pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.275756 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.275732    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-metrics-certs\") pod \"maas-controller-6c6d6b6d8d-kc22x\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") " pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.275877 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.275777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-webhook-cert\") pod \"maas-controller-6c6d6b6d8d-kc22x\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") " pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.281462 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.281431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l6h7\" (UniqueName: \"kubernetes.io/projected/486bee5f-d3b7-4133-a03a-4f75a4736f7d-kube-api-access-4l6h7\") pod \"maas-controller-6c6d6b6d8d-kc22x\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") " pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.401360 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.401324    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:48.576184 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.576146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-metrics-certs\") pod \"maas-controller-55d86fb7c8-jjf26\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") " pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:48.578627 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.578576    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-metrics-certs\") pod \"maas-controller-55d86fb7c8-jjf26\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") " pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:48.677208 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.676356    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-kc22x"]
Sep 11 01:07:48.809805 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:48.809769    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:49.083253 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:49.083225    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55d86fb7c8-jjf26"]
Sep 11 01:07:49.085306 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:07:49.085274    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8b21b39_6bcb_497c_ae07_953e32e45541.slice/crio-eb2c225efac077f20b7e64e3c1b2156a9ed0aa1b5f58748937a5bf847db0b3b5 WatchSource:0}: Error finding container eb2c225efac077f20b7e64e3c1b2156a9ed0aa1b5f58748937a5bf847db0b3b5: Status 404 returned error can't find the container with id eb2c225efac077f20b7e64e3c1b2156a9ed0aa1b5f58748937a5bf847db0b3b5
Sep 11 01:07:49.117320 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:49.117289    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55d86fb7c8-jjf26" event={"ID":"c8b21b39-6bcb-497c-ae07-953e32e45541","Type":"ContainerStarted","Data":"eb2c225efac077f20b7e64e3c1b2156a9ed0aa1b5f58748937a5bf847db0b3b5"}
Sep 11 01:07:49.118379 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:49.118352    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x" event={"ID":"486bee5f-d3b7-4133-a03a-4f75a4736f7d","Type":"ContainerStarted","Data":"d0a7c0c583fb140ff86154d7ae7c9e185247d2c4dbb86557110da62f1bdc6782"}
Sep 11 01:07:52.140635 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:52.140582    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55d86fb7c8-jjf26" event={"ID":"c8b21b39-6bcb-497c-ae07-953e32e45541","Type":"ContainerStarted","Data":"3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700"}
Sep 11 01:07:52.141062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:52.140662    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:07:52.142175 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:52.142153    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x" event={"ID":"486bee5f-d3b7-4133-a03a-4f75a4736f7d","Type":"ContainerStarted","Data":"b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6"}
Sep 11 01:07:52.142296 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:52.142280    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:07:52.159984 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:52.159931    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55d86fb7c8-jjf26" podStartSLOduration=3.089614704 podStartE2EDuration="5.159914843s" totalImagesPullingTime="2.070300139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:07:52.15776996 +0000 UTC m=+591.672370176" watchObservedRunningTime="2026-09-11 01:07:52.159914843 +0000 UTC m=+591.674515037"
Sep 11 01:07:52.177740 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:07:52.177690    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x" podStartSLOduration=1.711146695 podStartE2EDuration="4.177672898s" totalImagesPullingTime="2.466526203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:07:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:07:52.177432215 +0000 UTC m=+591.692032400" watchObservedRunningTime="2026-09-11 01:07:52.177672898 +0000 UTC m=+591.692273079"
Sep 11 01:08:03.149804 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.149774    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:08:03.151449 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.149851    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:08:03.212796 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.212767    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55d86fb7c8-jjf26"]
Sep 11 01:08:03.212990 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.212949    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55d86fb7c8-jjf26" podUID="c8b21b39-6bcb-497c-ae07-953e32e45541" containerName="manager" containerID="cri-o://3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700" gracePeriod=30
Sep 11 01:08:03.449614 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.449587    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:08:03.510748 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.510718    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-webhook-cert\") pod \"c8b21b39-6bcb-497c-ae07-953e32e45541\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") "
Sep 11 01:08:03.510905 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.510766    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lscj2\" (UniqueName: \"kubernetes.io/projected/c8b21b39-6bcb-497c-ae07-953e32e45541-kube-api-access-lscj2\") pod \"c8b21b39-6bcb-497c-ae07-953e32e45541\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") "
Sep 11 01:08:03.510905 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.510836    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-metrics-certs\") pod \"c8b21b39-6bcb-497c-ae07-953e32e45541\" (UID: \"c8b21b39-6bcb-497c-ae07-953e32e45541\") "
Sep 11 01:08:03.513129 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.513094    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c8b21b39-6bcb-497c-ae07-953e32e45541" (UID: "c8b21b39-6bcb-497c-ae07-953e32e45541"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:08:03.513129 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.513114    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c8b21b39-6bcb-497c-ae07-953e32e45541" (UID: "c8b21b39-6bcb-497c-ae07-953e32e45541"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:08:03.513129 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.513101    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8b21b39-6bcb-497c-ae07-953e32e45541-kube-api-access-lscj2" (OuterVolumeSpecName: "kube-api-access-lscj2") pod "c8b21b39-6bcb-497c-ae07-953e32e45541" (UID: "c8b21b39-6bcb-497c-ae07-953e32e45541"). InnerVolumeSpecName "kube-api-access-lscj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:08:03.611897 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.611857    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-webhook-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:08:03.611897 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.611893    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lscj2\" (UniqueName: \"kubernetes.io/projected/c8b21b39-6bcb-497c-ae07-953e32e45541-kube-api-access-lscj2\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:08:03.612105 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:03.611907    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b21b39-6bcb-497c-ae07-953e32e45541-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:08:04.213967 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:04.213934    2559 generic.go:356] "Generic (PLEG): container finished" podID="c8b21b39-6bcb-497c-ae07-953e32e45541" containerID="3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700" exitCode=0
Sep 11 01:08:04.214351 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:04.213976    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55d86fb7c8-jjf26"
Sep 11 01:08:04.214351 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:04.214015    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55d86fb7c8-jjf26" event={"ID":"c8b21b39-6bcb-497c-ae07-953e32e45541","Type":"ContainerDied","Data":"3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700"}
Sep 11 01:08:04.214351 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:04.214065    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55d86fb7c8-jjf26" event={"ID":"c8b21b39-6bcb-497c-ae07-953e32e45541","Type":"ContainerDied","Data":"eb2c225efac077f20b7e64e3c1b2156a9ed0aa1b5f58748937a5bf847db0b3b5"}
Sep 11 01:08:04.214351 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:04.214089    2559 scope.go:117] "RemoveContainer" containerID="3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700"
Sep 11 01:08:04.224164 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:04.224149    2559 scope.go:117] "RemoveContainer" containerID="3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700"
Sep 11 01:08:04.224419 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:08:04.224404    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700\": container with ID starting with 3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700 not found: ID does not exist" containerID="3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700"
Sep 11 01:08:04.224468 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:04.224427    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700"} err="failed to get container status \"3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700\": rpc error: code = NotFound desc = could not find container \"3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700\": container with ID starting with 3557b1421a5e6fb5fc421175ff225d3626ee827d315296766edc961c8fd7b700 not found: ID does not exist"
Sep 11 01:08:04.236119 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:04.236097    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55d86fb7c8-jjf26"]
Sep 11 01:08:04.241531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:04.241512    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55d86fb7c8-jjf26"]
Sep 11 01:08:05.109945 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:05.109915    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8b21b39-6bcb-497c-ae07-953e32e45541" path="/var/lib/kubelet/pods/c8b21b39-6bcb-497c-ae07-953e32e45541/volumes"
Sep 11 01:08:25.466064 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.465964    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"]
Sep 11 01:08:25.466561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.466392    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8b21b39-6bcb-497c-ae07-953e32e45541" containerName="manager"
Sep 11 01:08:25.466561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.466406    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8b21b39-6bcb-497c-ae07-953e32e45541" containerName="manager"
Sep 11 01:08:25.466561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.466469    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8b21b39-6bcb-497c-ae07-953e32e45541" containerName="manager"
Sep 11 01:08:25.469573 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.469553    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.473410 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.473391    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 01:08:25.473493 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.473460    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 01:08:25.473493 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.473470    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fdvhg\""
Sep 11 01:08:25.482460 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.482436    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"]
Sep 11 01:08:25.612693 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.612661    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-metrics-certs\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.612900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.612699    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7149ce46-ba29-4b77-8297-6b1325db3127-openshift-service-ca\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.612900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.612731    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-maas-api-tls\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.612900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.612779    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4j4x\" (UniqueName: \"kubernetes.io/projected/7149ce46-ba29-4b77-8297-6b1325db3127-kube-api-access-n4j4x\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.713474 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.713439    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4j4x\" (UniqueName: \"kubernetes.io/projected/7149ce46-ba29-4b77-8297-6b1325db3127-kube-api-access-n4j4x\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.713645 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.713519    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-metrics-certs\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.713645 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.713547    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7149ce46-ba29-4b77-8297-6b1325db3127-openshift-service-ca\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.713645 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.713578    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-maas-api-tls\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.713797 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:08:25.713700    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 11 01:08:25.713797 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:08:25.713783    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-maas-api-tls podName:7149ce46-ba29-4b77-8297-6b1325db3127 nodeName:}" failed. No retries permitted until 2026-09-11 01:08:26.213762325 +0000 UTC m=+625.728362485 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-maas-api-tls") pod "maas-api-7b7ff8bc87-5rmg4" (UID: "7149ce46-ba29-4b77-8297-6b1325db3127") : secret "maas-api-serving-cert" not found
Sep 11 01:08:25.714205 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.714180    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7149ce46-ba29-4b77-8297-6b1325db3127-openshift-service-ca\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.716119 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.716068    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-metrics-certs\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.724894 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.724861    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4j4x\" (UniqueName: \"kubernetes.io/projected/7149ce46-ba29-4b77-8297-6b1325db3127-kube-api-access-n4j4x\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:25.995847 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.995771    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-t27rf"]
Sep 11 01:08:25.999501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:25.999482    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf"
Sep 11 01:08:26.002274 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.002255    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rh479\""
Sep 11 01:08:26.002373 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.002261    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 01:08:26.010135 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.010111    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-t27rf"]
Sep 11 01:08:26.116981 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.116950    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8312527e-f390-4329-bd4c-e76f9cf8c530-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-t27rf\" (UID: \"8312527e-f390-4329-bd4c-e76f9cf8c530\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf"
Sep 11 01:08:26.117167 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.117031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8x78\" (UniqueName: \"kubernetes.io/projected/8312527e-f390-4329-bd4c-e76f9cf8c530-kube-api-access-m8x78\") pod \"payload-processing-5b54b7fbb7-t27rf\" (UID: \"8312527e-f390-4329-bd4c-e76f9cf8c530\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf"
Sep 11 01:08:26.217632 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.217576    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8x78\" (UniqueName: \"kubernetes.io/projected/8312527e-f390-4329-bd4c-e76f9cf8c530-kube-api-access-m8x78\") pod \"payload-processing-5b54b7fbb7-t27rf\" (UID: \"8312527e-f390-4329-bd4c-e76f9cf8c530\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf"
Sep 11 01:08:26.217843 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.217822    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-maas-api-tls\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:26.217927 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.217898    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8312527e-f390-4329-bd4c-e76f9cf8c530-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-t27rf\" (UID: \"8312527e-f390-4329-bd4c-e76f9cf8c530\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf"
Sep 11 01:08:26.218589 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.218563    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8312527e-f390-4329-bd4c-e76f9cf8c530-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-t27rf\" (UID: \"8312527e-f390-4329-bd4c-e76f9cf8c530\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf"
Sep 11 01:08:26.220207 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.220183    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-maas-api-tls\") pod \"maas-api-7b7ff8bc87-5rmg4\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") " pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:26.229825 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.229801    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8x78\" (UniqueName: \"kubernetes.io/projected/8312527e-f390-4329-bd4c-e76f9cf8c530-kube-api-access-m8x78\") pod \"payload-processing-5b54b7fbb7-t27rf\" (UID: \"8312527e-f390-4329-bd4c-e76f9cf8c530\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf"
Sep 11 01:08:26.311673 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.311641    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf"
Sep 11 01:08:26.379990 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.379961    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:26.583055 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.583028    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-t27rf"]
Sep 11 01:08:26.584698 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:08:26.584676    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8312527e_f390_4329_bd4c_e76f9cf8c530.slice/crio-5d3cac0b9047d0baa9d31afae2bfd7b74451c01d46a297d09ee85764ab07ecd6 WatchSource:0}: Error finding container 5d3cac0b9047d0baa9d31afae2bfd7b74451c01d46a297d09ee85764ab07ecd6: Status 404 returned error can't find the container with id 5d3cac0b9047d0baa9d31afae2bfd7b74451c01d46a297d09ee85764ab07ecd6
Sep 11 01:08:26.586522 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.586506    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 01:08:26.658454 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:26.658376    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"]
Sep 11 01:08:26.666358 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:08:26.666324    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7149ce46_ba29_4b77_8297_6b1325db3127.slice/crio-8e1d866a2d1a8d87bdf1634a9d7d0710be2ba4eb93fea915a6575bd16982f55f WatchSource:0}: Error finding container 8e1d866a2d1a8d87bdf1634a9d7d0710be2ba4eb93fea915a6575bd16982f55f: Status 404 returned error can't find the container with id 8e1d866a2d1a8d87bdf1634a9d7d0710be2ba4eb93fea915a6575bd16982f55f
Sep 11 01:08:27.367816 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:27.367725    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf" event={"ID":"8312527e-f390-4329-bd4c-e76f9cf8c530","Type":"ContainerStarted","Data":"5d3cac0b9047d0baa9d31afae2bfd7b74451c01d46a297d09ee85764ab07ecd6"}
Sep 11 01:08:27.370224 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:27.370193    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4" event={"ID":"7149ce46-ba29-4b77-8297-6b1325db3127","Type":"ContainerStarted","Data":"8e1d866a2d1a8d87bdf1634a9d7d0710be2ba4eb93fea915a6575bd16982f55f"}
Sep 11 01:08:29.385927 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:29.385851    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4" event={"ID":"7149ce46-ba29-4b77-8297-6b1325db3127","Type":"ContainerStarted","Data":"08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983"}
Sep 11 01:08:29.385927 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:29.385914    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:29.387599 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:29.387576    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf" event={"ID":"8312527e-f390-4329-bd4c-e76f9cf8c530","Type":"ContainerStarted","Data":"e455980d318752fbaf05f2fe3b99a7360cf36bf2464828db1652da668e036437"}
Sep 11 01:08:29.387728 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:29.387666    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf"
Sep 11 01:08:29.403038 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:29.402883    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4" podStartSLOduration=1.991563174 podStartE2EDuration="4.402865827s" totalImagesPullingTime="2.411302653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:08:29.401816078 +0000 UTC m=+628.916416272" watchObservedRunningTime="2026-09-11 01:08:29.402865827 +0000 UTC m=+628.917466010"
Sep 11 01:08:29.418615 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:29.418562    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf" podStartSLOduration=1.924995718 podStartE2EDuration="4.418549757s" totalImagesPullingTime="2.493554039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:08:29.416415123 +0000 UTC m=+628.931015307" watchObservedRunningTime="2026-09-11 01:08:29.418549757 +0000 UTC m=+628.933149939"
Sep 11 01:08:35.398975 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:35.398900    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:08:35.424998 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:35.424966    2559 generic.go:356] "Generic (PLEG): container finished" podID="be6b052f-904e-4764-950e-fb94ac2d1571" containerID="dc8d29db32170c75c04bc68678f55701f5fadcd9e137181c0207fc9508567594" exitCode=0
Sep 11 01:08:35.425174 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:35.425040    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st" event={"ID":"be6b052f-904e-4764-950e-fb94ac2d1571","Type":"ContainerDied","Data":"dc8d29db32170c75c04bc68678f55701f5fadcd9e137181c0207fc9508567594"}
Sep 11 01:08:35.425414 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:35.425388    2559 scope.go:117] "RemoveContainer" containerID="dc8d29db32170c75c04bc68678f55701f5fadcd9e137181c0207fc9508567594"
Sep 11 01:08:35.426623 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:35.426575    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:08:35.426719 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:35.426684    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:08:35.426760 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:35.426735    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:08:36.432672 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:36.432639    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st" event={"ID":"be6b052f-904e-4764-950e-fb94ac2d1571","Type":"ContainerStarted","Data":"760f5316567c836030fb5d0ddab89e7b0dcdff649c5688436231d91b596faef1"}
Sep 11 01:08:37.262531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:37.262492    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:08:37.263695 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:37.263662    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f72st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 11 01:08:37.263845 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:37.263728    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st" podUID="be6b052f-904e-4764-950e-fb94ac2d1571" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 11 01:08:38.262352 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:38.262302    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f72st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 11 01:08:38.262847 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:38.262373    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st" podUID="be6b052f-904e-4764-950e-fb94ac2d1571" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 11 01:08:39.262650 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:39.262268    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f72st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 11 01:08:39.262650 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:39.262335    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st" podUID="be6b052f-904e-4764-950e-fb94ac2d1571" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 11 01:08:40.262631 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:40.262570    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f72st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 11 01:08:40.262810 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:40.262671    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st" podUID="be6b052f-904e-4764-950e-fb94ac2d1571" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 11 01:08:40.395180 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:40.395148    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-t27rf"
Sep 11 01:08:40.819899 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:40.819868    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:08:41.264677 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:41.264646    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f72st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 11 01:08:41.265108 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:41.264704    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st" podUID="be6b052f-904e-4764-950e-fb94ac2d1571" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 11 01:08:42.265720 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:42.265687    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:08:42.266793 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:42.266770    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f72st"
Sep 11 01:08:47.693739 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.693702    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"]
Sep 11 01:08:47.696318 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.696298    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.699020 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.698997    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 01:08:47.700304 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.700284    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6tkxm\""
Sep 11 01:08:47.700304 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.700297    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 11 01:08:47.700466 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.700319    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 01:08:47.707097 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.707076    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"]
Sep 11 01:08:47.751356 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.751315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.751941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.751385    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.751941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.751420    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c523034c-228e-4c99-a96c-2b63e3ba25c5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.751941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.751568    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.751941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.751749    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.751941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.751838    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f6nb\" (UniqueName: \"kubernetes.io/projected/c523034c-228e-4c99-a96c-2b63e3ba25c5-kube-api-access-2f6nb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.852743 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.852705    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.852948 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.852757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c523034c-228e-4c99-a96c-2b63e3ba25c5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.852948 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.852818    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.852948 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.852897    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.852948 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.852933    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f6nb\" (UniqueName: \"kubernetes.io/projected/c523034c-228e-4c99-a96c-2b63e3ba25c5-kube-api-access-2f6nb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.853165 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.852959    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.853234 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.853180    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.853315 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.853290    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.853379 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.853338    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.855246 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.855223    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c523034c-228e-4c99-a96c-2b63e3ba25c5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.855352 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.855272    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c523034c-228e-4c99-a96c-2b63e3ba25c5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:47.861034 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:47.861013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f6nb\" (UniqueName: \"kubernetes.io/projected/c523034c-228e-4c99-a96c-2b63e3ba25c5-kube-api-access-2f6nb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg\" (UID: \"c523034c-228e-4c99-a96c-2b63e3ba25c5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:48.007561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:48.007471    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:48.283562 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:48.283537    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"]
Sep 11 01:08:48.285167 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:08:48.285132    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc523034c_228e_4c99_a96c_2b63e3ba25c5.slice/crio-c8bfa4c187f917acc8888185934db4132084f4d7d78936891d17b05e7269d911 WatchSource:0}: Error finding container c8bfa4c187f917acc8888185934db4132084f4d7d78936891d17b05e7269d911: Status 404 returned error can't find the container with id c8bfa4c187f917acc8888185934db4132084f4d7d78936891d17b05e7269d911
Sep 11 01:08:48.510761 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:48.510726    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg" event={"ID":"c523034c-228e-4c99-a96c-2b63e3ba25c5","Type":"ContainerStarted","Data":"c8bfa4c187f917acc8888185934db4132084f4d7d78936891d17b05e7269d911"}
Sep 11 01:08:49.518073 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:49.518032    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg" event={"ID":"c523034c-228e-4c99-a96c-2b63e3ba25c5","Type":"ContainerStarted","Data":"5ca5042b562fb4c09b57271b29fa4d798c6ac0b41713113a1a380a66d8d5a947"}
Sep 11 01:08:49.539663 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:49.539618    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg" podStartSLOduration=1.586906505 podStartE2EDuration="2.539587021s" totalImagesPullingTime="952.680516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:08:49.538261202 +0000 UTC m=+649.052861384" watchObservedRunningTime="2026-09-11 01:08:49.539587021 +0000 UTC m=+649.054187208"
Sep 11 01:08:58.008164 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:58.008125    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:58.008589 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:58.008190    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:58.020983 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:58.020953    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:08:58.585677 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:08:58.585644    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg"
Sep 11 01:09:16.973843 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:16.973810    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"]
Sep 11 01:09:16.979297 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:16.979266    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:16.981912 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:16.981886    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 01:09:16.988263 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:16.988239    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"]
Sep 11 01:09:17.019098 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.019070    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.019238 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.019115    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9377bffc-4a48-4e4e-9576-9c5f461f3805-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.019238 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.019135    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.019238 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.019157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.019238 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.019227    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd8fq\" (UniqueName: \"kubernetes.io/projected/9377bffc-4a48-4e4e-9576-9c5f461f3805-kube-api-access-zd8fq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.019376 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.019266    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.120414 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.120378    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd8fq\" (UniqueName: \"kubernetes.io/projected/9377bffc-4a48-4e4e-9576-9c5f461f3805-kube-api-access-zd8fq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.120633 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.120430    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.120633 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.120537    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.120826 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.120802    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9377bffc-4a48-4e4e-9576-9c5f461f3805-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.120906 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.120865    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.120968 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.120908    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.120968 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.120924    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.121175 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.121155    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.121236 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.121209    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.123121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.123100    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9377bffc-4a48-4e4e-9576-9c5f461f3805-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.123204 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.123186    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9377bffc-4a48-4e4e-9576-9c5f461f3805-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.130115 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.130094    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd8fq\" (UniqueName: \"kubernetes.io/projected/9377bffc-4a48-4e4e-9576-9c5f461f3805-kube-api-access-zd8fq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hs58k\" (UID: \"9377bffc-4a48-4e4e-9576-9c5f461f3805\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.290537 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.290496    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:17.568147 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.568120    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"]
Sep 11 01:09:17.568768 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:09:17.568744    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9377bffc_4a48_4e4e_9576_9c5f461f3805.slice/crio-a335c742225f858f366d6b0dde4605934393825faef235ff22ed99758e2424e3 WatchSource:0}: Error finding container a335c742225f858f366d6b0dde4605934393825faef235ff22ed99758e2424e3: Status 404 returned error can't find the container with id a335c742225f858f366d6b0dde4605934393825faef235ff22ed99758e2424e3
Sep 11 01:09:17.684480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:17.684436    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k" event={"ID":"9377bffc-4a48-4e4e-9576-9c5f461f3805","Type":"ContainerStarted","Data":"a335c742225f858f366d6b0dde4605934393825faef235ff22ed99758e2424e3"}
Sep 11 01:09:18.691621 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:18.691577    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k" event={"ID":"9377bffc-4a48-4e4e-9576-9c5f461f3805","Type":"ContainerStarted","Data":"3fed5977b0fc8e52249aa8957977786772add8b29407a4e5ae899981b3572130"}
Sep 11 01:09:18.723972 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:18.723916    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k" podStartSLOduration=2.5335783210000002 podStartE2EDuration="2.723897601s" totalImagesPullingTime="190.31928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:09:18.722779766 +0000 UTC m=+678.237379949" watchObservedRunningTime="2026-09-11 01:09:18.723897601 +0000 UTC m=+678.238497782"
Sep 11 01:09:27.290665 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:27.290600    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:27.291097 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:27.290696    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:27.303164 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:27.303132    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:09:27.758327 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:09:27.758238    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hs58k"
Sep 11 01:12:16.507713 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.507679    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"]
Sep 11 01:12:16.511523 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.511498    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.514391 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.514360    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-cd176e-openshift-default-dockercfg-bjq76\""
Sep 11 01:12:16.525720 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.525695    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"]
Sep 11 01:12:16.578806 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.578768    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"]
Sep 11 01:12:16.582431 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.582399    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.585118 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.585095    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e98d1e-openshift-default-dockercfg-gqnqz\""
Sep 11 01:12:16.595439 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.595412    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"]
Sep 11 01:12:16.597784 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.597750    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-data\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.597919 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.597805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istiod-ca-cert\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.597919 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.597856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-data\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.597919 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.597895    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istiod-ca-cert\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.598075 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.597970    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-certs\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.598075 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598009    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-token\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.598075 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598039    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-token\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.598223 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598088    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-socket\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.598223 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598136    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-credential-socket\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.598223 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-certs\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.598223 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598202    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ncjp\" (UniqueName: \"kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-kube-api-access-9ncjp\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.598429 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598243    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-credential-socket\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.598429 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598265    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-envoy\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.598429 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598309    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-socket\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.598429 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598334    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-envoy\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.598429 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598413    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-podinfo\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.598725 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598443    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-podinfo\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.598725 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.598470    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvjml\" (UniqueName: \"kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-kube-api-access-cvjml\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.666654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.666585    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"]
Sep 11 01:12:16.670366 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.670341    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.674274 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.674251    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-f4de10-openshift-default-dockercfg-t8j4l\""
Sep 11 01:12:16.693497 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.693447    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"]
Sep 11 01:12:16.699531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699492    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-socket\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.699717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699545    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-envoy\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.699717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699579    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-socket\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.699717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699622    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-podinfo\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.699717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699647    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-data\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.699717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699693    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-podinfo\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.700011 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699720    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvjml\" (UniqueName: \"kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-kube-api-access-cvjml\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.700011 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-envoy\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.700011 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699782    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-data\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.700011 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699813    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-certs\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.700011 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69ade794-505f-4668-9d61-3703316ccb1c-istio-podinfo\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.700011 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699880    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69ade794-505f-4668-9d61-3703316ccb1c-istiod-ca-cert\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.700011 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.699924    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istiod-ca-cert\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700009    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-data\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700039    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-socket\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700047    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istiod-ca-cert\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700135    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-certs\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700178    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-istio-token\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700207    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-token\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-token\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700280    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-socket\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700313    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-credential-socket\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700339    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-credential-socket\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kg9p\" (UniqueName: \"kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-kube-api-access-7kg9p\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700395    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-data\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.700427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-certs\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.701654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700448    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ncjp\" (UniqueName: \"kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-kube-api-access-9ncjp\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.701654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700494    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-credential-socket\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.701654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700526    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-envoy\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.701654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700722    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-data\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.701654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.700990    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istiod-ca-cert\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.702021 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.701753    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-socket\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.702086 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.702038    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istiod-ca-cert\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.702155 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.702085    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-credential-socket\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.702433 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.702409    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-certs\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.702862 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.702792    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-certs\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.703030 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.702955    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-credential-socket\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.704848 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.704805    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-envoy\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.705167 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.705147    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-podinfo\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.705306 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.705283    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-podinfo\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.705829 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.705802    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-envoy\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.709031 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.708977    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvjml\" (UniqueName: \"kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-kube-api-access-cvjml\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.711251 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.711212    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-token\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.711440 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.711396    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ncjp\" (UniqueName: \"kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-kube-api-access-9ncjp\") pod \"e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") " pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.711972 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.711947    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-token\") pod \"e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") " pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.801453 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801422    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-credential-socket\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.801650 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801470    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kg9p\" (UniqueName: \"kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-kube-api-access-7kg9p\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.801650 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801571    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-socket\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.801650 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801640    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-data\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.801782 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-envoy\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.801782 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801723    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-certs\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.801782 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801759    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69ade794-505f-4668-9d61-3703316ccb1c-istio-podinfo\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.801932 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801782    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69ade794-505f-4668-9d61-3703316ccb1c-istiod-ca-cert\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.801932 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-istio-token\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.802045 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801935    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-credential-socket\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.802045 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.801969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-socket\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.802045 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.802017    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-data\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.802199 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.802105    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-certs\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.802558 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.802541    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69ade794-505f-4668-9d61-3703316ccb1c-istiod-ca-cert\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.804080 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.804059    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-envoy\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.804325 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.804302    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69ade794-505f-4668-9d61-3703316ccb1c-istio-podinfo\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.810145 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.810119    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kg9p\" (UniqueName: \"kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-kube-api-access-7kg9p\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.810667 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.810646    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-istio-token\") pod \"e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") " pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:16.822777 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.822752    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:16.894984 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.894953    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:16.987713 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:16.987680    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:17.103126 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.103035    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"]
Sep 11 01:12:17.104047 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:12:17.104016    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cfdb29b_d87a_41c4_b7ca_9c3525724d1e.slice/crio-989e6e5516a35e9c9ff3e6e68090a2de0b32b8f8d1b44b9f099e81bf5c28c959 WatchSource:0}: Error finding container 989e6e5516a35e9c9ff3e6e68090a2de0b32b8f8d1b44b9f099e81bf5c28c959: Status 404 returned error can't find the container with id 989e6e5516a35e9c9ff3e6e68090a2de0b32b8f8d1b44b9f099e81bf5c28c959
Sep 11 01:12:17.106223 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.106189    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:17.106311 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.106254    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:17.106311 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.106284    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:17.177381 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.177339    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"]
Sep 11 01:12:17.178444 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:12:17.178417    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a63be8e_9c0a_49c4_9768_e5dde759b2bf.slice/crio-e7c0bf66f5ed79479886a5b783e08b0c30d9f8c2f17ca3c6a483bd76a0fe1cf0 WatchSource:0}: Error finding container e7c0bf66f5ed79479886a5b783e08b0c30d9f8c2f17ca3c6a483bd76a0fe1cf0: Status 404 returned error can't find the container with id e7c0bf66f5ed79479886a5b783e08b0c30d9f8c2f17ca3c6a483bd76a0fe1cf0
Sep 11 01:12:17.181625 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.181565    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:17.181732 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.181712    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:17.181779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.181758    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:17.265936 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.265904    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"]
Sep 11 01:12:17.266465 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:12:17.266439    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ade794_505f_4668_9d61_3703316ccb1c.slice/crio-b40cc579c1e8c2eb529bb40327d3f4385b53cb603adb89fba071dc15ab69bdef WatchSource:0}: Error finding container b40cc579c1e8c2eb529bb40327d3f4385b53cb603adb89fba071dc15ab69bdef: Status 404 returned error can't find the container with id b40cc579c1e8c2eb529bb40327d3f4385b53cb603adb89fba071dc15ab69bdef
Sep 11 01:12:17.269726 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.269640    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:17.269726 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.269722    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:17.269892 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.269763    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:17.718079 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.718036    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" event={"ID":"69ade794-505f-4668-9d61-3703316ccb1c","Type":"ContainerStarted","Data":"c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944"}
Sep 11 01:12:17.718527 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.718106    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" event={"ID":"69ade794-505f-4668-9d61-3703316ccb1c","Type":"ContainerStarted","Data":"b40cc579c1e8c2eb529bb40327d3f4385b53cb603adb89fba071dc15ab69bdef"}
Sep 11 01:12:17.719944 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.719918    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" event={"ID":"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e","Type":"ContainerStarted","Data":"55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802"}
Sep 11 01:12:17.720079 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.719965    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" event={"ID":"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e","Type":"ContainerStarted","Data":"989e6e5516a35e9c9ff3e6e68090a2de0b32b8f8d1b44b9f099e81bf5c28c959"}
Sep 11 01:12:17.721468 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.721444    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" event={"ID":"1a63be8e-9c0a-49c4-9768-e5dde759b2bf","Type":"ContainerStarted","Data":"db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc"}
Sep 11 01:12:17.721580 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.721488    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" event={"ID":"1a63be8e-9c0a-49c4-9768-e5dde759b2bf","Type":"ContainerStarted","Data":"e7c0bf66f5ed79479886a5b783e08b0c30d9f8c2f17ca3c6a483bd76a0fe1cf0"}
Sep 11 01:12:17.747677 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.747619    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" podStartSLOduration=1.7475863440000001 podStartE2EDuration="1.747586344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:17.743868902 +0000 UTC m=+857.258469084" watchObservedRunningTime="2026-09-11 01:12:17.747586344 +0000 UTC m=+857.262186527"
Sep 11 01:12:17.775927 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.775877    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" podStartSLOduration=1.7758604980000001 podStartE2EDuration="1.775860498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:17.773839815 +0000 UTC m=+857.288440000" watchObservedRunningTime="2026-09-11 01:12:17.775860498 +0000 UTC m=+857.290460691"
Sep 11 01:12:17.809677 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.809627    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" podStartSLOduration=1.809598848 podStartE2EDuration="1.809598848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:17.806752283 +0000 UTC m=+857.321352468" watchObservedRunningTime="2026-09-11 01:12:17.809598848 +0000 UTC m=+857.324199034"
Sep 11 01:12:17.823412 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.823384    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:17.895395 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.895366    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:17.988138 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:17.988054    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:18.828935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:18.828905    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:18.900868 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:18.900840    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:18.993862 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:18.993835    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:19.735356 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:19.735317    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:19.735558 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:19.735411    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:19.735558 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:19.735439    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:19.735829 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:19.735811    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:19.735876 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:19.735865    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:19.736029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:19.736013    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:21.102172 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.102133    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"]
Sep 11 01:12:21.109693 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.109670    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.112443 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.112419    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e98d1e-serving-cert\""
Sep 11 01:12:21.122143 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.121836    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"]
Sep 11 01:12:21.148772 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.148739    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhhht\" (UniqueName: \"kubernetes.io/projected/74872bf5-d708-4f26-8527-57077dd9813b-kube-api-access-lhhht\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.148982 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.148788    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-metrics-certs\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.148982 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.148809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74872bf5-d708-4f26-8527-57077dd9813b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.149122 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.149017    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-maas-api-tls\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.250498 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.250461    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-maas-api-tls\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.250721 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.250512    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhhht\" (UniqueName: \"kubernetes.io/projected/74872bf5-d708-4f26-8527-57077dd9813b-kube-api-access-lhhht\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.250721 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:12:21.250640    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-serving-cert: secret "maas-api-e2e-inf-a-e98d1e-serving-cert" not found
Sep 11 01:12:21.250721 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.250653    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-metrics-certs\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.250721 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.250695    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74872bf5-d708-4f26-8527-57077dd9813b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.250956 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:12:21.250841    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-maas-api-tls podName:74872bf5-d708-4f26-8527-57077dd9813b nodeName:}" failed. No retries permitted until 2026-09-11 01:12:21.750817499 +0000 UTC m=+861.265417666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-maas-api-tls") pod "maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg" (UID: "74872bf5-d708-4f26-8527-57077dd9813b") : secret "maas-api-e2e-inf-a-e98d1e-serving-cert" not found
Sep 11 01:12:21.251294 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.251269    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74872bf5-d708-4f26-8527-57077dd9813b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.253336 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.253314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-metrics-certs\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.259636 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.259581    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhhht\" (UniqueName: \"kubernetes.io/projected/74872bf5-d708-4f26-8527-57077dd9813b-kube-api-access-lhhht\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.756948 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.756916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-maas-api-tls\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:21.759561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:21.759531    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-maas-api-tls\") pod \"maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:22.026979 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:22.026949    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:22.349136 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:22.348973    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"]
Sep 11 01:12:22.352366 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:12:22.352331    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74872bf5_d708_4f26_8527_57077dd9813b.slice/crio-07b498028663a3031941e62a46413965109a1eb51a583c1727aa7a3ccf5b0b5b WatchSource:0}: Error finding container 07b498028663a3031941e62a46413965109a1eb51a583c1727aa7a3ccf5b0b5b: Status 404 returned error can't find the container with id 07b498028663a3031941e62a46413965109a1eb51a583c1727aa7a3ccf5b0b5b
Sep 11 01:12:22.754872 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:22.754050    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg" event={"ID":"74872bf5-d708-4f26-8527-57077dd9813b","Type":"ContainerStarted","Data":"07b498028663a3031941e62a46413965109a1eb51a583c1727aa7a3ccf5b0b5b"}
Sep 11 01:12:23.491826 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.491785    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"]
Sep 11 01:12:23.496021 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.495996    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.498502 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.498471    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-cd176e-serving-cert\""
Sep 11 01:12:23.508519 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.508490    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"]
Sep 11 01:12:23.579733 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.579695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-metrics-certs\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.579914 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.579772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/091d0047-2df1-4ea3-a8a3-8635bc79ca61-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.579995 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.579940    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.580051 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.579993    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v22q\" (UniqueName: \"kubernetes.io/projected/091d0047-2df1-4ea3-a8a3-8635bc79ca61-kube-api-access-4v22q\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.681107 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.681065    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/091d0047-2df1-4ea3-a8a3-8635bc79ca61-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.681294 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.681187    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.681294 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.681219    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v22q\" (UniqueName: \"kubernetes.io/projected/091d0047-2df1-4ea3-a8a3-8635bc79ca61-kube-api-access-4v22q\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.681294 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.681244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-metrics-certs\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.681457 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:12:23.681332    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-serving-cert: secret "maas-api-e2e-worker-w4-cd176e-serving-cert" not found
Sep 11 01:12:23.681457 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:12:23.681398    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-maas-api-tls podName:091d0047-2df1-4ea3-a8a3-8635bc79ca61 nodeName:}" failed. No retries permitted until 2026-09-11 01:12:24.181383494 +0000 UTC m=+863.695983655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-maas-api-tls") pod "maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j" (UID: "091d0047-2df1-4ea3-a8a3-8635bc79ca61") : secret "maas-api-e2e-worker-w4-cd176e-serving-cert" not found
Sep 11 01:12:23.681806 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.681784    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/091d0047-2df1-4ea3-a8a3-8635bc79ca61-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.683639 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.683597    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-metrics-certs\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.689557 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.689536    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v22q\" (UniqueName: \"kubernetes.io/projected/091d0047-2df1-4ea3-a8a3-8635bc79ca61-kube-api-access-4v22q\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:23.762068 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.761975    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg" event={"ID":"74872bf5-d708-4f26-8527-57077dd9813b","Type":"ContainerStarted","Data":"fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913"}
Sep 11 01:12:23.762298 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.762073    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:23.782674 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:23.782619    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg" podStartSLOduration=2.374603865 podStartE2EDuration="2.782590088s" totalImagesPullingTime="407.986223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:23.77939174 +0000 UTC m=+863.293991924" watchObservedRunningTime="2026-09-11 01:12:23.782590088 +0000 UTC m=+863.297190270"
Sep 11 01:12:24.187666 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:24.187581    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:24.190126 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:24.190103    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:24.408852 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:24.408812    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:24.684701 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:24.684671    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"]
Sep 11 01:12:24.685521 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:12:24.685500    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091d0047_2df1_4ea3_a8a3_8635bc79ca61.slice/crio-41577ff931e2eb9823a48b65c0de2ce4543bb81ca6056efc0a6935cd67a72b8d WatchSource:0}: Error finding container 41577ff931e2eb9823a48b65c0de2ce4543bb81ca6056efc0a6935cd67a72b8d: Status 404 returned error can't find the container with id 41577ff931e2eb9823a48b65c0de2ce4543bb81ca6056efc0a6935cd67a72b8d
Sep 11 01:12:24.769620 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:24.769571    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j" event={"ID":"091d0047-2df1-4ea3-a8a3-8635bc79ca61","Type":"ContainerStarted","Data":"41577ff931e2eb9823a48b65c0de2ce4543bb81ca6056efc0a6935cd67a72b8d"}
Sep 11 01:12:25.401815 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.401728    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"]
Sep 11 01:12:25.405415 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.405393    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.409485 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.409464    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-2c1e69-openshift-default-dockercfg-qlnpd\""
Sep 11 01:12:25.420089 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.420062    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"]
Sep 11 01:12:25.501521 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.501482    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-socket\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.501727 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.501543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-data\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.501727 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.501645    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istiod-ca-cert\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.501727 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.501703    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-token\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.501947 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.501745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-credential-socket\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.501947 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.501851    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-envoy\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.501947 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.501892    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-certs\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.501947 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.501935    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ffp5\" (UniqueName: \"kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-kube-api-access-2ffp5\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.502115 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.501993    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-podinfo\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603084 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603054    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-envoy\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603260 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603093    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-certs\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603260 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603121    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ffp5\" (UniqueName: \"kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-kube-api-access-2ffp5\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603260 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603147    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-podinfo\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603260 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603184    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-socket\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603260 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603217    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-data\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603260 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603261    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istiod-ca-cert\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603504 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-token\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603504 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603345    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-credential-socket\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603661 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603638    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-certs\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603761 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603660    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-data\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603761 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603706    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-credential-socket\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.603869 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.603769    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-socket\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.604092 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.604070    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istiod-ca-cert\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.605639 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.605599    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-envoy\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.606010 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.605990    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-podinfo\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.611096 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.611075    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ffp5\" (UniqueName: \"kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-kube-api-access-2ffp5\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.611198 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.611118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-token\") pod \"e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") " pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.716173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.716085    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:25.785119 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.784111    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j" event={"ID":"091d0047-2df1-4ea3-a8a3-8635bc79ca61","Type":"ContainerStarted","Data":"3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc"}
Sep 11 01:12:25.785119 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.785038    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:25.820805 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:25.820756    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j" podStartSLOduration=2.502254744 podStartE2EDuration="2.820739771s" totalImagesPullingTime="318.485027ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:25.818641451 +0000 UTC m=+865.333241635" watchObservedRunningTime="2026-09-11 01:12:25.820739771 +0000 UTC m=+865.335340000"
Sep 11 01:12:26.011885 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:12:26.011845    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bca294c_83b5_46f9_9fec_fe7c59a5003c.slice/crio-76600dbd6983c6b36f62053de5a49d3a35dcc85f9feda922b104210962924177 WatchSource:0}: Error finding container 76600dbd6983c6b36f62053de5a49d3a35dcc85f9feda922b104210962924177: Status 404 returned error can't find the container with id 76600dbd6983c6b36f62053de5a49d3a35dcc85f9feda922b104210962924177
Sep 11 01:12:26.012421 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:26.012344    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"]
Sep 11 01:12:26.014348 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:26.014309    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:26.014432 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:26.014397    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:26.014488 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:26.014444    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:26.794917 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:26.794880    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" event={"ID":"8bca294c-83b5-46f9-9fec-fe7c59a5003c","Type":"ContainerStarted","Data":"b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085"}
Sep 11 01:12:26.795327 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:26.794939    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" event={"ID":"8bca294c-83b5-46f9-9fec-fe7c59a5003c","Type":"ContainerStarted","Data":"76600dbd6983c6b36f62053de5a49d3a35dcc85f9feda922b104210962924177"}
Sep 11 01:12:26.817139 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:26.817087    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" podStartSLOduration=1.817070562 podStartE2EDuration="1.817070562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:26.814184204 +0000 UTC m=+866.328784387" watchObservedRunningTime="2026-09-11 01:12:26.817070562 +0000 UTC m=+866.331670754"
Sep 11 01:12:27.717159 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:27.717125    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:27.722415 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:27.722388    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:27.800622 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:27.800582    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:27.801351 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:27.801335    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:29.788904 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:29.788865    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:12:32.814704 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:32.814668    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:12:34.449499 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.449455    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"]
Sep 11 01:12:34.454809 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.454779    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.464541 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.464169    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fd82e780-serving-cert\""
Sep 11 01:12:34.504728 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.501669    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"]
Sep 11 01:12:34.531496 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.531456    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-maas-api-tls\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.531854 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.531817    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-metrics-certs\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.532118 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.532101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b876ead-22b7-4f35-9910-09c5e1bebfa6-openshift-service-ca\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.532293 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.532270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvmkh\" (UniqueName: \"kubernetes.io/projected/3b876ead-22b7-4f35-9910-09c5e1bebfa6-kube-api-access-fvmkh\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.633589 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.633551    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-maas-api-tls\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.633777 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.633652    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-metrics-certs\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.633777 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.633716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b876ead-22b7-4f35-9910-09c5e1bebfa6-openshift-service-ca\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.633777 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.633747    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvmkh\" (UniqueName: \"kubernetes.io/projected/3b876ead-22b7-4f35-9910-09c5e1bebfa6-kube-api-access-fvmkh\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.634022 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:12:34.634006    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-serving-cert: secret "maas-api-e2e-ait-fd82e780-serving-cert" not found
Sep 11 01:12:34.634173 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:12:34.634154    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-maas-api-tls podName:3b876ead-22b7-4f35-9910-09c5e1bebfa6 nodeName:}" failed. No retries permitted until 2026-09-11 01:12:35.13413456 +0000 UTC m=+874.648734723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-maas-api-tls") pod "maas-api-e2e-ait-fd82e780-58848cc858-gc6gd" (UID: "3b876ead-22b7-4f35-9910-09c5e1bebfa6") : secret "maas-api-e2e-ait-fd82e780-serving-cert" not found
Sep 11 01:12:34.634567 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.634519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b876ead-22b7-4f35-9910-09c5e1bebfa6-openshift-service-ca\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.636811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.636786    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-metrics-certs\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:34.644171 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:34.644147    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvmkh\" (UniqueName: \"kubernetes.io/projected/3b876ead-22b7-4f35-9910-09c5e1bebfa6-kube-api-access-fvmkh\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:35.147727 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:35.147694    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-maas-api-tls\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:35.150200 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:35.150170    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-maas-api-tls\") pod \"maas-api-e2e-ait-fd82e780-58848cc858-gc6gd\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:35.388881 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:35.388779    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:35.697661 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:35.696816    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"]
Sep 11 01:12:35.777853 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:35.777800    2559 patch_prober.go:28] interesting pod/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 01:12:35.778005 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:35.777879    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 01:12:35.875769 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:35.875693    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd" event={"ID":"3b876ead-22b7-4f35-9910-09c5e1bebfa6","Type":"ContainerStarted","Data":"31e0d4e5d9c61c60988d5d36239c1c87b4183cc5de5a2607e05854081c032261"}
Sep 11 01:12:36.887113 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:36.887071    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd" event={"ID":"3b876ead-22b7-4f35-9910-09c5e1bebfa6","Type":"ContainerStarted","Data":"dbf55aeddf943ddf0ae4916365689025576ea2ed3dfe49f43a05cc086c3cae38"}
Sep 11 01:12:36.887577 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:36.887202    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:36.909086 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:36.909025    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd" podStartSLOduration=2.47835112 podStartE2EDuration="2.909006226s" totalImagesPullingTime="430.655106ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:36.904129137 +0000 UTC m=+876.418729322" watchObservedRunningTime="2026-09-11 01:12:36.909006226 +0000 UTC m=+876.423606408"
Sep 11 01:12:38.903858 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.903822    2559 generic.go:356] "Generic (PLEG): container finished" podID="69ade794-505f-4668-9d61-3703316ccb1c" containerID="c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944" exitCode=0
Sep 11 01:12:38.904314 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.903895    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" event={"ID":"69ade794-505f-4668-9d61-3703316ccb1c","Type":"ContainerDied","Data":"c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944"}
Sep 11 01:12:38.904417 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.904401    2559 scope.go:117] "RemoveContainer" containerID="c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944"
Sep 11 01:12:38.905812 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.905774    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:38.905909 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.905815    2559 generic.go:356] "Generic (PLEG): container finished" podID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerID="55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802" exitCode=0
Sep 11 01:12:38.905909 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.905876    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:38.905909 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.905891    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" event={"ID":"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e","Type":"ContainerDied","Data":"55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802"}
Sep 11 01:12:38.906056 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.905927    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:38.906266 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.906248    2559 scope.go:117] "RemoveContainer" containerID="55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802"
Sep 11 01:12:38.907402 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.907376    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:38.907514 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.907451    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:38.907514 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.907503    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:38.907736 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.907718    2559 generic.go:356] "Generic (PLEG): container finished" podID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerID="db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc" exitCode=0
Sep 11 01:12:38.907811 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.907788    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" event={"ID":"1a63be8e-9c0a-49c4-9768-e5dde759b2bf","Type":"ContainerDied","Data":"db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc"}
Sep 11 01:12:38.908074 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.908061    2559 scope.go:117] "RemoveContainer" containerID="db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc"
Sep 11 01:12:38.909190 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.909129    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:38.909253 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.909199    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:38.909313 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:38.909247    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:39.915983 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:39.915942    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" event={"ID":"69ade794-505f-4668-9d61-3703316ccb1c","Type":"ContainerStarted","Data":"f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0"}
Sep 11 01:12:39.918425 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:39.918400    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" event={"ID":"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e","Type":"ContainerStarted","Data":"0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3"}
Sep 11 01:12:39.920926 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:39.920904    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" event={"ID":"1a63be8e-9c0a-49c4-9768-e5dde759b2bf","Type":"ContainerStarted","Data":"816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff"}
Sep 11 01:12:39.987890 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:39.987852    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:40.823382 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:40.823338    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:40.823578 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:40.823479    2559 patch_prober.go:28] interesting pod/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 11 01:12:40.823578 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:40.823524    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 11 01:12:40.896243 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:40.896198    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:40.896453 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:40.896340    2559 patch_prober.go:28] interesting pod/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 11 01:12:40.896453 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:40.896383    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 11 01:12:40.994627 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:40.989396    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 01:12:40.994627 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:40.989483    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 01:12:41.823485 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:41.823439    2559 patch_prober.go:28] interesting pod/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 11 01:12:41.823724 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:41.823509    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 11 01:12:41.896551 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:41.896508    2559 patch_prober.go:28] interesting pod/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 11 01:12:41.897300 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:41.896862    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 11 01:12:41.989020 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:41.988979    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 01:12:41.989341 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:41.989295    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 01:12:42.824192 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:42.824145    2559 patch_prober.go:28] interesting pod/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 11 01:12:42.824693 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:42.824220    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 11 01:12:42.895683 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:42.895637    2559 patch_prober.go:28] interesting pod/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 11 01:12:42.895886 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:42.895720    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 11 01:12:42.905537 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:42.905486    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:12:42.990517 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:42.990486    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 01:12:42.990807 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:42.990776    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 01:12:43.823644 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:43.823580    2559 patch_prober.go:28] interesting pod/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 11 01:12:43.823963 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:43.823928    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 11 01:12:43.895593 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:43.895554    2559 patch_prober.go:28] interesting pod/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 11 01:12:43.896222 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:43.896193    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 11 01:12:43.993584 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:43.993544    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 01:12:43.993917 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:43.993871    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 01:12:44.823884 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:44.823846    2559 patch_prober.go:28] interesting pod/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 11 01:12:44.824213 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:44.824161    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 11 01:12:44.896062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:44.896023    2559 patch_prober.go:28] interesting pod/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 11 01:12:44.896636 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:44.896582    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 11 01:12:44.989786 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:44.989320    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 01:12:44.989786 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:44.989387    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 01:12:45.512827 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.512779    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"]
Sep 11 01:12:45.519798 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.518953    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.522180 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.522130    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-10a215-openshift-default-dockercfg-lhjdv\""
Sep 11 01:12:45.575133 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.575088    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"]
Sep 11 01:12:45.702480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.701965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-socket\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.702480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.702031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-podinfo\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.702480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.702083    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-token\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.702480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.702157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-envoy\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.702480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.702188    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-credential-socket\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.702480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.702234    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-data\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.702480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.702260    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-certs\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.702480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.702303    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be3fa414-e05b-4a3a-aedf-571a724ca39f-istiod-ca-cert\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.702480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.702353    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf2vm\" (UniqueName: \"kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-kube-api-access-gf2vm\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.803491 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.803450    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-token\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.803739 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.803541    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-envoy\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.803739 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.803569    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-credential-socket\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.803739 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.803623    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-data\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.803739 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.803650    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-certs\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.803739 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.803696    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be3fa414-e05b-4a3a-aedf-571a724ca39f-istiod-ca-cert\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.804016 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.803740    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf2vm\" (UniqueName: \"kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-kube-api-access-gf2vm\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.804016 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.803792    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-socket\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.804016 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.803836    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-podinfo\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.805241 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.805213    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-certs\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.805768 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.805543    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-credential-socket\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.805768 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.805719    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-socket\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.806646 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.806628    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be3fa414-e05b-4a3a-aedf-571a724ca39f-istiod-ca-cert\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.807258 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.807240    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-data\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.812144 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.811882    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-envoy\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.812790 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.812661    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-podinfo\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.815449 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.815390    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-token\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.817171 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.817131    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf2vm\" (UniqueName: \"kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-kube-api-access-gf2vm\") pod \"e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") " pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.844435 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.844102    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:45.897417 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.896964    2559 patch_prober.go:28] interesting pod/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 11 01:12:45.897417 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:45.897028    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 11 01:12:46.036651 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.017475    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 11 01:12:46.036651 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.017556    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 11 01:12:46.177575 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.177481    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:46.179543 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.179522    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:46.181531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.181494    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:12:46.294653 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.294618    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"]
Sep 11 01:12:46.308230 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.301477    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:46.308230 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.301663    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:46.308230 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.301732    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:46.901913 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.901885    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:46.903023 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.902964    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:46.903964 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.903947    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:12:46.993417 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.993389    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:46.993819 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.993791    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:46.994812 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:46.994792    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:12:47.083637 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:47.083504    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" event={"ID":"be3fa414-e05b-4a3a-aedf-571a724ca39f","Type":"ContainerStarted","Data":"a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4"}
Sep 11 01:12:47.083637 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:47.083571    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" event={"ID":"be3fa414-e05b-4a3a-aedf-571a724ca39f","Type":"ContainerStarted","Data":"854bd6e25800f575b6210a0b98ff7c286c92080e59fba5c35bdf9da6e958de79"}
Sep 11 01:12:47.105881 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:47.105822    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" podStartSLOduration=2.105799896 podStartE2EDuration="2.105799896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:47.104438582 +0000 UTC m=+886.619038764" watchObservedRunningTime="2026-09-11 01:12:47.105799896 +0000 UTC m=+886.620400080"
Sep 11 01:12:47.845187 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:47.845150    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:47.855028 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:47.854995    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:48.089938 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:48.089908    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:48.090701 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:48.090675    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:12:50.303887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.303854    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"]
Sep 11 01:12:50.307056 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.307042    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.309530 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.309513    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-10a215-serving-cert\""
Sep 11 01:12:50.317740 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.317714    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"]
Sep 11 01:12:50.367748 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.367705    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vrww\" (UniqueName: \"kubernetes.io/projected/a3bbbf11-216e-4615-85ef-d10e967c8c23-kube-api-access-9vrww\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.367905 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.367770    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-metrics-certs\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.367905 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.367815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-maas-api-tls\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.367995 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.367904    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbbf11-216e-4615-85ef-d10e967c8c23-openshift-service-ca\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.469665 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.469568    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbbf11-216e-4615-85ef-d10e967c8c23-openshift-service-ca\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.469846 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.469706    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vrww\" (UniqueName: \"kubernetes.io/projected/a3bbbf11-216e-4615-85ef-d10e967c8c23-kube-api-access-9vrww\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.469846 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.469738    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-metrics-certs\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.469846 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.469772    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-maas-api-tls\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.470311 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.470284    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbbf11-216e-4615-85ef-d10e967c8c23-openshift-service-ca\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.472187 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.472164    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-metrics-certs\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.472284 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.472207    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-maas-api-tls\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.480495 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.480470    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vrww\" (UniqueName: \"kubernetes.io/projected/a3bbbf11-216e-4615-85ef-d10e967c8c23-kube-api-access-9vrww\") pod \"maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.649058 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.649026    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:50.933256 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:50.933231    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"]
Sep 11 01:12:50.934566 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:12:50.934543    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3bbbf11_216e_4615_85ef_d10e967c8c23.slice/crio-0628d5935737d56421b24b218f1bb274a6d699072f221de29dc1d97e847d1a67 WatchSource:0}: Error finding container 0628d5935737d56421b24b218f1bb274a6d699072f221de29dc1d97e847d1a67: Status 404 returned error can't find the container with id 0628d5935737d56421b24b218f1bb274a6d699072f221de29dc1d97e847d1a67
Sep 11 01:12:51.114412 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:51.114382    2559 generic.go:356] "Generic (PLEG): container finished" podID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerID="b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085" exitCode=0
Sep 11 01:12:51.114626 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:51.114456    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" event={"ID":"8bca294c-83b5-46f9-9fec-fe7c59a5003c","Type":"ContainerDied","Data":"b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085"}
Sep 11 01:12:51.114943 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:51.114921    2559 scope.go:117] "RemoveContainer" containerID="b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085"
Sep 11 01:12:51.115873 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:51.115848    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64" event={"ID":"a3bbbf11-216e-4615-85ef-d10e967c8c23","Type":"ContainerStarted","Data":"0628d5935737d56421b24b218f1bb274a6d699072f221de29dc1d97e847d1a67"}
Sep 11 01:12:51.116159 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:51.116132    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:51.116211 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:51.116198    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:51.116251 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:51.116233    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:12:52.129873 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:52.129831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" event={"ID":"8bca294c-83b5-46f9-9fec-fe7c59a5003c","Type":"ContainerStarted","Data":"ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128"}
Sep 11 01:12:52.132444 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:52.132406    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64" event={"ID":"a3bbbf11-216e-4615-85ef-d10e967c8c23","Type":"ContainerStarted","Data":"6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0"}
Sep 11 01:12:52.133066 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:52.133041    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:12:52.178298 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:52.178233    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64" podStartSLOduration=1.868708695 podStartE2EDuration="2.178209705s" totalImagesPullingTime="309.50101ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:12:52.176167728 +0000 UTC m=+891.690767913" watchObservedRunningTime="2026-09-11 01:12:52.178209705 +0000 UTC m=+891.692809888"
Sep 11 01:12:52.717224 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:52.717187    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:52.717393 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:52.717371    2559 patch_prober.go:28] interesting pod/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 11 01:12:52.717459 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:52.717436    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 11 01:12:53.716653 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:53.716617    2559 patch_prober.go:28] interesting pod/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 11 01:12:53.717148 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:53.716681    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 11 01:12:54.717242 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:54.717211    2559 patch_prober.go:28] interesting pod/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 11 01:12:54.717699 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:54.717265    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 11 01:12:55.716785 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:55.716748    2559 patch_prober.go:28] interesting pod/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 11 01:12:55.716984 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:55.716805    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 11 01:12:56.716624 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:56.716578    2559 patch_prober.go:28] interesting pod/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 11 01:12:56.717022 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:56.716667    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 11 01:12:57.781196 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:57.781159    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:57.781671 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:57.781637    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:57.782909 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:57.782890    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:12:59.157583 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:12:59.157555    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:13:05.082429 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.082341    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"]
Sep 11 01:13:05.082927 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.082650    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd" podUID="3b876ead-22b7-4f35-9910-09c5e1bebfa6" containerName="maas-api" containerID="cri-o://dbf55aeddf943ddf0ae4916365689025576ea2ed3dfe49f43a05cc086c3cae38" gracePeriod=30
Sep 11 01:13:05.235583 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.235438    2559 generic.go:356] "Generic (PLEG): container finished" podID="3b876ead-22b7-4f35-9910-09c5e1bebfa6" containerID="dbf55aeddf943ddf0ae4916365689025576ea2ed3dfe49f43a05cc086c3cae38" exitCode=0
Sep 11 01:13:05.235583 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.235507    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd" event={"ID":"3b876ead-22b7-4f35-9910-09c5e1bebfa6","Type":"ContainerDied","Data":"dbf55aeddf943ddf0ae4916365689025576ea2ed3dfe49f43a05cc086c3cae38"}
Sep 11 01:13:05.407813 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.407674    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:13:05.570931 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.570897    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-metrics-certs\") pod \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") "
Sep 11 01:13:05.571131 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.571002    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-maas-api-tls\") pod \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") "
Sep 11 01:13:05.571131 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.571033    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b876ead-22b7-4f35-9910-09c5e1bebfa6-openshift-service-ca\") pod \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") "
Sep 11 01:13:05.571131 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.571065    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvmkh\" (UniqueName: \"kubernetes.io/projected/3b876ead-22b7-4f35-9910-09c5e1bebfa6-kube-api-access-fvmkh\") pod \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\" (UID: \"3b876ead-22b7-4f35-9910-09c5e1bebfa6\") "
Sep 11 01:13:05.571386 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.571342    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b876ead-22b7-4f35-9910-09c5e1bebfa6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b876ead-22b7-4f35-9910-09c5e1bebfa6" (UID: "3b876ead-22b7-4f35-9910-09c5e1bebfa6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:13:05.573310 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.573283    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b876ead-22b7-4f35-9910-09c5e1bebfa6-kube-api-access-fvmkh" (OuterVolumeSpecName: "kube-api-access-fvmkh") pod "3b876ead-22b7-4f35-9910-09c5e1bebfa6" (UID: "3b876ead-22b7-4f35-9910-09c5e1bebfa6"). InnerVolumeSpecName "kube-api-access-fvmkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:13:05.573386 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.573320    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3b876ead-22b7-4f35-9910-09c5e1bebfa6" (UID: "3b876ead-22b7-4f35-9910-09c5e1bebfa6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:13:05.573836 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.573819    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b876ead-22b7-4f35-9910-09c5e1bebfa6" (UID: "3b876ead-22b7-4f35-9910-09c5e1bebfa6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:13:05.673061 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.672972    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:13:05.673061 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.673002    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b876ead-22b7-4f35-9910-09c5e1bebfa6-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:13:05.673061 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.673012    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvmkh\" (UniqueName: \"kubernetes.io/projected/3b876ead-22b7-4f35-9910-09c5e1bebfa6-kube-api-access-fvmkh\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:13:05.673061 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:05.673025    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b876ead-22b7-4f35-9910-09c5e1bebfa6-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:13:06.246700 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:06.246659    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd" event={"ID":"3b876ead-22b7-4f35-9910-09c5e1bebfa6","Type":"ContainerDied","Data":"31e0d4e5d9c61c60988d5d36239c1c87b4183cc5de5a2607e05854081c032261"}
Sep 11 01:13:06.247199 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:06.246740    2559 scope.go:117] "RemoveContainer" containerID="dbf55aeddf943ddf0ae4916365689025576ea2ed3dfe49f43a05cc086c3cae38"
Sep 11 01:13:06.247199 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:06.246914    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"
Sep 11 01:13:06.288539 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:06.288509    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"]
Sep 11 01:13:06.292332 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:06.292309    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd82e780-58848cc858-gc6gd"]
Sep 11 01:13:07.111196 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:07.111162    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b876ead-22b7-4f35-9910-09c5e1bebfa6" path="/var/lib/kubelet/pods/3b876ead-22b7-4f35-9910-09c5e1bebfa6/volumes"
Sep 11 01:13:09.057901 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.057857    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"]
Sep 11 01:13:09.058552 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.058537    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b876ead-22b7-4f35-9910-09c5e1bebfa6" containerName="maas-api"
Sep 11 01:13:09.058651 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.058554    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b876ead-22b7-4f35-9910-09c5e1bebfa6" containerName="maas-api"
Sep 11 01:13:09.058699 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.058673    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b876ead-22b7-4f35-9910-09c5e1bebfa6" containerName="maas-api"
Sep 11 01:13:09.062859 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.062832    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.065499 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.065476    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-0b6ae428-serving-cert\""
Sep 11 01:13:09.071495 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.071462    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"]
Sep 11 01:13:09.112427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.112394    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-metrics-certs\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.112597 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.112535    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a030505-c98f-4897-b2e9-6d525567ea8c-openshift-service-ca\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.112662 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.112627    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-maas-api-tls\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.112706 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.112677    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq4nt\" (UniqueName: \"kubernetes.io/projected/6a030505-c98f-4897-b2e9-6d525567ea8c-kube-api-access-wq4nt\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.214110 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.214080    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-metrics-certs\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.214254 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.214143    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a030505-c98f-4897-b2e9-6d525567ea8c-openshift-service-ca\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.214254 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.214184    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-maas-api-tls\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.214254 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.214212    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq4nt\" (UniqueName: \"kubernetes.io/projected/6a030505-c98f-4897-b2e9-6d525567ea8c-kube-api-access-wq4nt\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.214370 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:13:09.214318    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-serving-cert: secret "maas-api-e2e-migrate-0b6ae428-serving-cert" not found
Sep 11 01:13:09.214419 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:13:09.214409    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-maas-api-tls podName:6a030505-c98f-4897-b2e9-6d525567ea8c nodeName:}" failed. No retries permitted until 2026-09-11 01:13:09.714387499 +0000 UTC m=+909.228987689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-maas-api-tls") pod "maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6" (UID: "6a030505-c98f-4897-b2e9-6d525567ea8c") : secret "maas-api-e2e-migrate-0b6ae428-serving-cert" not found
Sep 11 01:13:09.215000 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.214970    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a030505-c98f-4897-b2e9-6d525567ea8c-openshift-service-ca\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.216711 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.216690    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-metrics-certs\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.223513 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.223492    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq4nt\" (UniqueName: \"kubernetes.io/projected/6a030505-c98f-4897-b2e9-6d525567ea8c-kube-api-access-wq4nt\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.720154 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.720115    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-maas-api-tls\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.722661 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.722557    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-maas-api-tls\") pod \"maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:09.979208 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:09.979119    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:10.257013 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:10.256983    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"]
Sep 11 01:13:10.258675 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:13:10.258639    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a030505_c98f_4897_b2e9_6d525567ea8c.slice/crio-528a56a5e255c51b6af7fa0366e317e7cfeb5390eb634552a46448d02897ac6b WatchSource:0}: Error finding container 528a56a5e255c51b6af7fa0366e317e7cfeb5390eb634552a46448d02897ac6b: Status 404 returned error can't find the container with id 528a56a5e255c51b6af7fa0366e317e7cfeb5390eb634552a46448d02897ac6b
Sep 11 01:13:10.279647 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:10.279618    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6" event={"ID":"6a030505-c98f-4897-b2e9-6d525567ea8c","Type":"ContainerStarted","Data":"528a56a5e255c51b6af7fa0366e317e7cfeb5390eb634552a46448d02897ac6b"}
Sep 11 01:13:11.287382 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:11.287339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6" event={"ID":"6a030505-c98f-4897-b2e9-6d525567ea8c","Type":"ContainerStarted","Data":"4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223"}
Sep 11 01:13:11.287899 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:11.287426    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:11.289132 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:11.289109    2559 generic.go:356] "Generic (PLEG): container finished" podID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerID="a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4" exitCode=0
Sep 11 01:13:11.289239 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:11.289144    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" event={"ID":"be3fa414-e05b-4a3a-aedf-571a724ca39f","Type":"ContainerDied","Data":"a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4"}
Sep 11 01:13:11.289501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:11.289490    2559 scope.go:117] "RemoveContainer" containerID="a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4"
Sep 11 01:13:11.290588 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:11.290561    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:13:11.290672 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:11.290657    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:13:11.290712 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:11.290693    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:13:11.308157 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:11.308111    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6" podStartSLOduration=2.005828001 podStartE2EDuration="2.308093724s" totalImagesPullingTime="302.265723ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:13:11.306458751 +0000 UTC m=+910.821058934" watchObservedRunningTime="2026-09-11 01:13:11.308093724 +0000 UTC m=+910.822693909"
Sep 11 01:13:12.304980 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:12.304936    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" event={"ID":"be3fa414-e05b-4a3a-aedf-571a724ca39f","Type":"ContainerStarted","Data":"5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2"}
Sep 11 01:13:12.845529 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:12.845384    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:13:12.845765 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:12.845732    2559 patch_prober.go:28] interesting pod/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 11 01:13:12.845823 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:12.845792    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 11 01:13:13.845226 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:13.845191    2559 patch_prober.go:28] interesting pod/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 11 01:13:13.845824 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:13.845790    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 11 01:13:14.845330 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:14.845292    2559 patch_prober.go:28] interesting pod/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 11 01:13:14.845953 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:14.845921    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 11 01:13:15.844997 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:15.844956    2559 patch_prober.go:28] interesting pod/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 11 01:13:15.845297 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:15.845261    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 11 01:13:16.845752 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:16.845704    2559 patch_prober.go:28] interesting pod/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 11 01:13:16.846256 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:16.845776    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 11 01:13:17.310588 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:17.310561    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:17.848855 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:17.848825    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:13:17.849285 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:17.849235    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:13:17.849997 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:17.849980    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:13:40.459668 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.459630    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"]
Sep 11 01:13:40.460215 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.459869    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6" podUID="6a030505-c98f-4897-b2e9-6d525567ea8c" containerName="maas-api" containerID="cri-o://4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223" gracePeriod=30
Sep 11 01:13:40.710040 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.709982    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:40.776537 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.776505    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a030505-c98f-4897-b2e9-6d525567ea8c-openshift-service-ca\") pod \"6a030505-c98f-4897-b2e9-6d525567ea8c\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") "
Sep 11 01:13:40.776537 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.776541    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-maas-api-tls\") pod \"6a030505-c98f-4897-b2e9-6d525567ea8c\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") "
Sep 11 01:13:40.776746 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.776573    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-metrics-certs\") pod \"6a030505-c98f-4897-b2e9-6d525567ea8c\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") "
Sep 11 01:13:40.776789 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.776765    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wq4nt\" (UniqueName: \"kubernetes.io/projected/6a030505-c98f-4897-b2e9-6d525567ea8c-kube-api-access-wq4nt\") pod \"6a030505-c98f-4897-b2e9-6d525567ea8c\" (UID: \"6a030505-c98f-4897-b2e9-6d525567ea8c\") "
Sep 11 01:13:40.777031 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.777004    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a030505-c98f-4897-b2e9-6d525567ea8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6a030505-c98f-4897-b2e9-6d525567ea8c" (UID: "6a030505-c98f-4897-b2e9-6d525567ea8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:13:40.777137 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.777118    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a030505-c98f-4897-b2e9-6d525567ea8c-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:13:40.778637 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.778598    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6a030505-c98f-4897-b2e9-6d525567ea8c" (UID: "6a030505-c98f-4897-b2e9-6d525567ea8c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:13:40.778731 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.778715    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6a030505-c98f-4897-b2e9-6d525567ea8c" (UID: "6a030505-c98f-4897-b2e9-6d525567ea8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:13:40.778864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.778841    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a030505-c98f-4897-b2e9-6d525567ea8c-kube-api-access-wq4nt" (OuterVolumeSpecName: "kube-api-access-wq4nt") pod "6a030505-c98f-4897-b2e9-6d525567ea8c" (UID: "6a030505-c98f-4897-b2e9-6d525567ea8c"). InnerVolumeSpecName "kube-api-access-wq4nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:13:40.878022 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.877986    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq4nt\" (UniqueName: \"kubernetes.io/projected/6a030505-c98f-4897-b2e9-6d525567ea8c-kube-api-access-wq4nt\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:13:40.878022 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.878017    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:13:40.878022 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:40.878030    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a030505-c98f-4897-b2e9-6d525567ea8c-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:13:41.486004 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:41.485918    2559 generic.go:356] "Generic (PLEG): container finished" podID="6a030505-c98f-4897-b2e9-6d525567ea8c" containerID="4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223" exitCode=0
Sep 11 01:13:41.486004 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:41.485964    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"
Sep 11 01:13:41.486446 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:41.486003    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6" event={"ID":"6a030505-c98f-4897-b2e9-6d525567ea8c","Type":"ContainerDied","Data":"4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223"}
Sep 11 01:13:41.486446 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:41.486059    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6" event={"ID":"6a030505-c98f-4897-b2e9-6d525567ea8c","Type":"ContainerDied","Data":"528a56a5e255c51b6af7fa0366e317e7cfeb5390eb634552a46448d02897ac6b"}
Sep 11 01:13:41.486446 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:41.486073    2559 scope.go:117] "RemoveContainer" containerID="4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223"
Sep 11 01:13:41.495678 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:41.495662    2559 scope.go:117] "RemoveContainer" containerID="4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223"
Sep 11 01:13:41.495938 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:13:41.495920    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223\": container with ID starting with 4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223 not found: ID does not exist" containerID="4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223"
Sep 11 01:13:41.495989 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:41.495945    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223"} err="failed to get container status \"4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223\": rpc error: code = NotFound desc = could not find container \"4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223\": container with ID starting with 4af9965352c5cf9733d047be987cafd54b56237be956ad9e296a4bd8b9459223 not found: ID does not exist"
Sep 11 01:13:41.505668 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:41.505627    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"]
Sep 11 01:13:41.508866 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:41.508845    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0b6ae428-d4b49b947-z7kl6"]
Sep 11 01:13:43.111260 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:43.111227    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a030505-c98f-4897-b2e9-6d525567ea8c" path="/var/lib/kubelet/pods/6a030505-c98f-4897-b2e9-6d525567ea8c/volumes"
Sep 11 01:13:45.249549 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.249517    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"]
Sep 11 01:13:45.250319 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.250290    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a030505-c98f-4897-b2e9-6d525567ea8c" containerName="maas-api"
Sep 11 01:13:45.250319 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.250318    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a030505-c98f-4897-b2e9-6d525567ea8c" containerName="maas-api"
Sep 11 01:13:45.250477 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.250456    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a030505-c98f-4897-b2e9-6d525567ea8c" containerName="maas-api"
Sep 11 01:13:45.253304 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.253284    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.255819 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.255792    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9516b91d-serving-cert\""
Sep 11 01:13:45.265952 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.265923    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"]
Sep 11 01:13:45.332531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.332497    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-maas-api-tls\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.332694 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.332534    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn7pk\" (UniqueName: \"kubernetes.io/projected/477e83cf-5852-44f6-80ea-75c4cc8a3c92-kube-api-access-mn7pk\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.332694 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.332575    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/477e83cf-5852-44f6-80ea-75c4cc8a3c92-openshift-service-ca\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.332794 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.332731    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-metrics-certs\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.433668 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.433631    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-metrics-certs\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.433857 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.433692    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-maas-api-tls\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.433857 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.433724    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn7pk\" (UniqueName: \"kubernetes.io/projected/477e83cf-5852-44f6-80ea-75c4cc8a3c92-kube-api-access-mn7pk\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.433857 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.433781    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/477e83cf-5852-44f6-80ea-75c4cc8a3c92-openshift-service-ca\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.433857 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:13:45.433833    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-serving-cert: secret "maas-api-e2e-ait-9516b91d-serving-cert" not found
Sep 11 01:13:45.434113 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:13:45.433901    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-maas-api-tls podName:477e83cf-5852-44f6-80ea-75c4cc8a3c92 nodeName:}" failed. No retries permitted until 2026-09-11 01:13:45.933884635 +0000 UTC m=+945.448484794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-maas-api-tls") pod "maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p" (UID: "477e83cf-5852-44f6-80ea-75c4cc8a3c92") : secret "maas-api-e2e-ait-9516b91d-serving-cert" not found
Sep 11 01:13:45.434472 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.434448    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/477e83cf-5852-44f6-80ea-75c4cc8a3c92-openshift-service-ca\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.436140 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.436116    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-metrics-certs\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.446290 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.446264    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn7pk\" (UniqueName: \"kubernetes.io/projected/477e83cf-5852-44f6-80ea-75c4cc8a3c92-kube-api-access-mn7pk\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.940621 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.940579    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-maas-api-tls\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:45.943047 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:45.943028    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-maas-api-tls\") pod \"maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:46.165463 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:46.165431    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:46.460866 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:46.460833    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"]
Sep 11 01:13:46.461566 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:13:46.461541    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477e83cf_5852_44f6_80ea_75c4cc8a3c92.slice/crio-cde3eca8ffffc5733f2f2974c761cd3a2742b3e4a8b06df3199092f2da687a4a WatchSource:0}: Error finding container cde3eca8ffffc5733f2f2974c761cd3a2742b3e4a8b06df3199092f2da687a4a: Status 404 returned error can't find the container with id cde3eca8ffffc5733f2f2974c761cd3a2742b3e4a8b06df3199092f2da687a4a
Sep 11 01:13:46.462987 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:46.462969    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 01:13:46.520516 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:46.520484    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p" event={"ID":"477e83cf-5852-44f6-80ea-75c4cc8a3c92","Type":"ContainerStarted","Data":"cde3eca8ffffc5733f2f2974c761cd3a2742b3e4a8b06df3199092f2da687a4a"}
Sep 11 01:13:47.528372 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:47.528339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p" event={"ID":"477e83cf-5852-44f6-80ea-75c4cc8a3c92","Type":"ContainerStarted","Data":"28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad"}
Sep 11 01:13:47.528824 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:47.528501    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:13:47.547518 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:47.547472    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p" podStartSLOduration=2.083861795 podStartE2EDuration="2.547457114s" totalImagesPullingTime="463.595319ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:13:47.546466312 +0000 UTC m=+947.061066495" watchObservedRunningTime="2026-09-11 01:13:47.547457114 +0000 UTC m=+947.062057340"
Sep 11 01:13:53.540195 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:13:53.540165    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:14:15.756405 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:15.756371    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"]
Sep 11 01:14:15.757001 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:15.756581    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p" podUID="477e83cf-5852-44f6-80ea-75c4cc8a3c92" containerName="maas-api" containerID="cri-o://28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad" gracePeriod=30
Sep 11 01:14:16.005361 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.005330    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:14:16.061032 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.061002    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-metrics-certs\") pod \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") "
Sep 11 01:14:16.061194 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.061082    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-maas-api-tls\") pod \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") "
Sep 11 01:14:16.061194 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.061163    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/477e83cf-5852-44f6-80ea-75c4cc8a3c92-openshift-service-ca\") pod \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") "
Sep 11 01:14:16.061312 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.061197    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn7pk\" (UniqueName: \"kubernetes.io/projected/477e83cf-5852-44f6-80ea-75c4cc8a3c92-kube-api-access-mn7pk\") pod \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\" (UID: \"477e83cf-5852-44f6-80ea-75c4cc8a3c92\") "
Sep 11 01:14:16.061531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.061502    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/477e83cf-5852-44f6-80ea-75c4cc8a3c92-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "477e83cf-5852-44f6-80ea-75c4cc8a3c92" (UID: "477e83cf-5852-44f6-80ea-75c4cc8a3c92"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:16.061664 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.061577    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/477e83cf-5852-44f6-80ea-75c4cc8a3c92-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:16.063286 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.063258    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "477e83cf-5852-44f6-80ea-75c4cc8a3c92" (UID: "477e83cf-5852-44f6-80ea-75c4cc8a3c92"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:14:16.063374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.063310    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/477e83cf-5852-44f6-80ea-75c4cc8a3c92-kube-api-access-mn7pk" (OuterVolumeSpecName: "kube-api-access-mn7pk") pod "477e83cf-5852-44f6-80ea-75c4cc8a3c92" (UID: "477e83cf-5852-44f6-80ea-75c4cc8a3c92"). InnerVolumeSpecName "kube-api-access-mn7pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:14:16.063374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.063328    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "477e83cf-5852-44f6-80ea-75c4cc8a3c92" (UID: "477e83cf-5852-44f6-80ea-75c4cc8a3c92"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:14:16.169237 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.169204    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn7pk\" (UniqueName: \"kubernetes.io/projected/477e83cf-5852-44f6-80ea-75c4cc8a3c92-kube-api-access-mn7pk\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:16.170755 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.170728    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:16.170936 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.170923    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/477e83cf-5852-44f6-80ea-75c4cc8a3c92-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:16.712956 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.712925    2559 generic.go:356] "Generic (PLEG): container finished" podID="477e83cf-5852-44f6-80ea-75c4cc8a3c92" containerID="28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad" exitCode=0
Sep 11 01:14:16.713154 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.712974    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"
Sep 11 01:14:16.713154 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.713011    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p" event={"ID":"477e83cf-5852-44f6-80ea-75c4cc8a3c92","Type":"ContainerDied","Data":"28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad"}
Sep 11 01:14:16.713154 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.713066    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p" event={"ID":"477e83cf-5852-44f6-80ea-75c4cc8a3c92","Type":"ContainerDied","Data":"cde3eca8ffffc5733f2f2974c761cd3a2742b3e4a8b06df3199092f2da687a4a"}
Sep 11 01:14:16.713154 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.713088    2559 scope.go:117] "RemoveContainer" containerID="28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad"
Sep 11 01:14:16.726623 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.726585    2559 scope.go:117] "RemoveContainer" containerID="28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad"
Sep 11 01:14:16.727025 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:14:16.726992    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad\": container with ID starting with 28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad not found: ID does not exist" containerID="28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad"
Sep 11 01:14:16.727111 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.727026    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad"} err="failed to get container status \"28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad\": rpc error: code = NotFound desc = could not find container \"28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad\": container with ID starting with 28eced7e376f2ac46cf0d670056cd1b4cd6f1fd04157ecddddea94d6ced32aad not found: ID does not exist"
Sep 11 01:14:16.739572 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.739549    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"]
Sep 11 01:14:16.742394 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:16.742374    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9516b91d-56cbbdd9bf-nf49p"]
Sep 11 01:14:17.115076 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:17.115033    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="477e83cf-5852-44f6-80ea-75c4cc8a3c92" path="/var/lib/kubelet/pods/477e83cf-5852-44f6-80ea-75c4cc8a3c92/volumes"
Sep 11 01:14:17.964510 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:17.964477    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"]
Sep 11 01:14:17.964982 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:17.964968    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="477e83cf-5852-44f6-80ea-75c4cc8a3c92" containerName="maas-api"
Sep 11 01:14:17.965030 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:17.964983    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="477e83cf-5852-44f6-80ea-75c4cc8a3c92" containerName="maas-api"
Sep 11 01:14:17.965082 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:17.965073    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="477e83cf-5852-44f6-80ea-75c4cc8a3c92" containerName="maas-api"
Sep 11 01:14:17.968184 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:17.968164    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:17.970648 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:17.970593    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6573d049-openshift-default-dockercfg-p2ckc\""
Sep 11 01:14:17.979993 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:17.979967    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"]
Sep 11 01:14:18.093309 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.093268    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istiod-ca-cert\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.093700 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.093661    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk4lk\" (UniqueName: \"kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-kube-api-access-gk4lk\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.093935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.093919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-token\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.094102 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.094088    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-data\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.094238 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.094223    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-socket\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.094412 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.094397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-certs\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.094556 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.094538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-podinfo\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.094751 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.094732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-credential-socket\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.094893 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.094878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-envoy\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.196469 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196436    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-token\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.196961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196485    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-data\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.196961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196509    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-socket\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.196961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-certs\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.196961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196578    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-podinfo\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.196961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196672    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-credential-socket\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.196961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196710    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-envoy\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.196961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196811    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istiod-ca-cert\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.196961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4lk\" (UniqueName: \"kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-kube-api-access-gk4lk\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.196961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196938    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-data\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.197420 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.196986    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-socket\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.197420 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.197240    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-certs\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.197420 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.197307    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-credential-socket\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.197684 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.197663    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istiod-ca-cert\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.199024 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.198998    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-envoy\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.199171 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.199151    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-podinfo\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.205038 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.205016    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk4lk\" (UniqueName: \"kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-kube-api-access-gk4lk\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.205125 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.205044    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-token\") pod \"e2e-derive-6573d049-openshift-default-7d86df964-t6x4s\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") " pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.279954 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.279922    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:18.556805 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:14:18.556776    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad3a4b7_6362_4859_88b4_8f94a14a6f83.slice/crio-cc0517721bbb255e475e96a49d3b340bb01b2ee0458c9873f3448c185562fb66 WatchSource:0}: Error finding container cc0517721bbb255e475e96a49d3b340bb01b2ee0458c9873f3448c185562fb66: Status 404 returned error can't find the container with id cc0517721bbb255e475e96a49d3b340bb01b2ee0458c9873f3448c185562fb66
Sep 11 01:14:18.557125 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.557101    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"]
Sep 11 01:14:18.559466 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.559433    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:14:18.559555 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.559495    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:14:18.559555 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.559538    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:14:18.737204 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.737169    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" event={"ID":"5ad3a4b7-6362-4859-88b4-8f94a14a6f83","Type":"ContainerStarted","Data":"117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d"}
Sep 11 01:14:18.737375 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.737253    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" event={"ID":"5ad3a4b7-6362-4859-88b4-8f94a14a6f83","Type":"ContainerStarted","Data":"cc0517721bbb255e475e96a49d3b340bb01b2ee0458c9873f3448c185562fb66"}
Sep 11 01:14:18.759029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:18.758975    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" podStartSLOduration=1.758959682 podStartE2EDuration="1.758959682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:14:18.756082087 +0000 UTC m=+978.270682264" watchObservedRunningTime="2026-09-11 01:14:18.758959682 +0000 UTC m=+978.273559924"
Sep 11 01:14:19.280280 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:19.280237    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:19.285287 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:19.285262    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:19.743737 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:19.743655    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:19.744492 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:19.744473    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:38.870006 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:38.869974    2559 generic.go:356] "Generic (PLEG): container finished" podID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerID="117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d" exitCode=0
Sep 11 01:14:38.870423 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:38.870047    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" event={"ID":"5ad3a4b7-6362-4859-88b4-8f94a14a6f83","Type":"ContainerDied","Data":"117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d"}
Sep 11 01:14:38.870497 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:38.870484    2559 scope.go:117] "RemoveContainer" containerID="117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d"
Sep 11 01:14:38.871645 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:38.871590    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:14:38.871702 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:38.871693    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:14:38.871738 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:38.871726    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:14:39.877864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:39.877831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" event={"ID":"5ad3a4b7-6362-4859-88b4-8f94a14a6f83","Type":"ContainerStarted","Data":"e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff"}
Sep 11 01:14:40.280392 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:40.280354    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:40.280674 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:40.280649    2559 patch_prober.go:28] interesting pod/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 11 01:14:40.280781 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:40.280709    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 11 01:14:41.280278 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:41.280231    2559 patch_prober.go:28] interesting pod/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 11 01:14:41.280775 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:41.280293    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 11 01:14:42.280739 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:42.280701    2559 patch_prober.go:28] interesting pod/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 11 01:14:42.281135 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:42.280753    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 11 01:14:43.281199 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:43.281163    2559 patch_prober.go:28] interesting pod/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 11 01:14:43.281588 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:43.281220    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 11 01:14:44.280309 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:44.280274    2559 patch_prober.go:28] interesting pod/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 11 01:14:44.280480 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:44.280343    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 11 01:14:45.280304 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:45.280263    2559 patch_prober.go:28] interesting pod/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 11 01:14:45.280702 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:45.280332    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 11 01:14:46.284272 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:46.284241    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:46.284818 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:46.284634    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:46.285538 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:46.285504    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:52.168724 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:52.168692    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"]
Sep 11 01:14:52.169325 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:52.168951    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy" containerID="cri-o://e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff" gracePeriod=30
Sep 11 01:14:57.421669 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.421645    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:57.441485 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441425    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-credential-socket\") pod \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") "
Sep 11 01:14:57.441598 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441487    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istiod-ca-cert\") pod \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") "
Sep 11 01:14:57.441598 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441505    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-data\") pod \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") "
Sep 11 01:14:57.441598 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441526    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-certs\") pod \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") "
Sep 11 01:14:57.441598 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441551    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-podinfo\") pod \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") "
Sep 11 01:14:57.441598 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441591    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-envoy\") pod \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") "
Sep 11 01:14:57.441863 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441639    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gk4lk\" (UniqueName: \"kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-kube-api-access-gk4lk\") pod \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") "
Sep 11 01:14:57.441863 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441699    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-socket\") pod \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") "
Sep 11 01:14:57.441863 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441717    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5ad3a4b7-6362-4859-88b4-8f94a14a6f83" (UID: "5ad3a4b7-6362-4859-88b4-8f94a14a6f83"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:14:57.441863 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441804    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-token\") pod \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\" (UID: \"5ad3a4b7-6362-4859-88b4-8f94a14a6f83\") "
Sep 11 01:14:57.441863 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441826    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-data" (OuterVolumeSpecName: "istio-data") pod "5ad3a4b7-6362-4859-88b4-8f94a14a6f83" (UID: "5ad3a4b7-6362-4859-88b4-8f94a14a6f83"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:14:57.441863 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441818    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5ad3a4b7-6362-4859-88b4-8f94a14a6f83" (UID: "5ad3a4b7-6362-4859-88b4-8f94a14a6f83"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:14:57.442190 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.441987    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5ad3a4b7-6362-4859-88b4-8f94a14a6f83" (UID: "5ad3a4b7-6362-4859-88b4-8f94a14a6f83"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:14:57.442190 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.442102    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5ad3a4b7-6362-4859-88b4-8f94a14a6f83" (UID: "5ad3a4b7-6362-4859-88b4-8f94a14a6f83"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:14:57.442299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.442236    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:57.442299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.442251    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:57.442299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.442259    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:57.442299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.442268    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:57.442299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.442276    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:57.445242 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.445104    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5ad3a4b7-6362-4859-88b4-8f94a14a6f83" (UID: "5ad3a4b7-6362-4859-88b4-8f94a14a6f83"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:14:57.446005 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.445130    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-token" (OuterVolumeSpecName: "istio-token") pod "5ad3a4b7-6362-4859-88b4-8f94a14a6f83" (UID: "5ad3a4b7-6362-4859-88b4-8f94a14a6f83"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:14:57.446005 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.445410    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-kube-api-access-gk4lk" (OuterVolumeSpecName: "kube-api-access-gk4lk") pod "5ad3a4b7-6362-4859-88b4-8f94a14a6f83" (UID: "5ad3a4b7-6362-4859-88b4-8f94a14a6f83"). InnerVolumeSpecName "kube-api-access-gk4lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:14:57.446493 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.446444    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5ad3a4b7-6362-4859-88b4-8f94a14a6f83" (UID: "5ad3a4b7-6362-4859-88b4-8f94a14a6f83"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:14:57.543900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.543866    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:57.543900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.543892    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:57.543900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.543902    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:57.543900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.543910    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gk4lk\" (UniqueName: \"kubernetes.io/projected/5ad3a4b7-6362-4859-88b4-8f94a14a6f83-kube-api-access-gk4lk\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:14:57.993852 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.993813    2559 generic.go:356] "Generic (PLEG): container finished" podID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerID="e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff" exitCode=0
Sep 11 01:14:57.994060 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.993866    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"
Sep 11 01:14:57.994060 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.993903    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" event={"ID":"5ad3a4b7-6362-4859-88b4-8f94a14a6f83","Type":"ContainerDied","Data":"e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff"}
Sep 11 01:14:57.994060 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.993973    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s" event={"ID":"5ad3a4b7-6362-4859-88b4-8f94a14a6f83","Type":"ContainerDied","Data":"cc0517721bbb255e475e96a49d3b340bb01b2ee0458c9873f3448c185562fb66"}
Sep 11 01:14:57.994060 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:57.994006    2559 scope.go:117] "RemoveContainer" containerID="e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff"
Sep 11 01:14:58.006069 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:58.006051    2559 scope.go:117] "RemoveContainer" containerID="117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d"
Sep 11 01:14:58.016018 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:58.015998    2559 scope.go:117] "RemoveContainer" containerID="e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff"
Sep 11 01:14:58.016299 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:14:58.016280    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff\": container with ID starting with e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff not found: ID does not exist" containerID="e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff"
Sep 11 01:14:58.016342 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:58.016310    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff"} err="failed to get container status \"e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff\": rpc error: code = NotFound desc = could not find container \"e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff\": container with ID starting with e72eb7160d24c9d3774e2565048876f706d80ea1566ab02b1b3c03584dfbebff not found: ID does not exist"
Sep 11 01:14:58.016342 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:58.016334    2559 scope.go:117] "RemoveContainer" containerID="117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d"
Sep 11 01:14:58.016581 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:14:58.016559    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d\": container with ID starting with 117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d not found: ID does not exist" containerID="117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d"
Sep 11 01:14:58.016669 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:58.016593    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d"} err="failed to get container status \"117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d\": rpc error: code = NotFound desc = could not find container \"117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d\": container with ID starting with 117c6d72108e79df0cdd3193ae1bfd4cd3863a9db4329dc3de86ca8f31b82f5d not found: ID does not exist"
Sep 11 01:14:58.020986 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:58.020947    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"]
Sep 11 01:14:58.024698 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:58.024678    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6573d049-openshift-default-7d86df964-t6x4s"]
Sep 11 01:14:59.111029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:14:59.110999    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" path="/var/lib/kubelet/pods/5ad3a4b7-6362-4859-88b4-8f94a14a6f83/volumes"
Sep 11 01:15:04.972858 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:04.972822    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"]
Sep 11 01:15:04.973280 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:04.973034    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy" containerID="cri-o://ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128" gracePeriod=30
Sep 11 01:15:10.232327 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.232299    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:15:10.382658 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.382587    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istiod-ca-cert\") pod \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") "
Sep 11 01:15:10.382658 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.382638    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-podinfo\") pod \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") "
Sep 11 01:15:10.382944 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.382671    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-socket\") pod \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") "
Sep 11 01:15:10.382944 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.382721    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-envoy\") pod \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") "
Sep 11 01:15:10.382944 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.382796    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-credential-socket\") pod \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") "
Sep 11 01:15:10.382944 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.382847    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-data\") pod \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") "
Sep 11 01:15:10.382944 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.382875    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-certs\") pod \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") "
Sep 11 01:15:10.382944 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.382901    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ffp5\" (UniqueName: \"kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-kube-api-access-2ffp5\") pod \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") "
Sep 11 01:15:10.382944 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.382933    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-token\") pod \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\" (UID: \"8bca294c-83b5-46f9-9fec-fe7c59a5003c\") "
Sep 11 01:15:10.383307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.382976    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8bca294c-83b5-46f9-9fec-fe7c59a5003c" (UID: "8bca294c-83b5-46f9-9fec-fe7c59a5003c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:15:10.383307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.383020    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8bca294c-83b5-46f9-9fec-fe7c59a5003c" (UID: "8bca294c-83b5-46f9-9fec-fe7c59a5003c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:15:10.383307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.383196    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8bca294c-83b5-46f9-9fec-fe7c59a5003c" (UID: "8bca294c-83b5-46f9-9fec-fe7c59a5003c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:15:10.383307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.383230    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8bca294c-83b5-46f9-9fec-fe7c59a5003c" (UID: "8bca294c-83b5-46f9-9fec-fe7c59a5003c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:15:10.383501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.383342    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-data" (OuterVolumeSpecName: "istio-data") pod "8bca294c-83b5-46f9-9fec-fe7c59a5003c" (UID: "8bca294c-83b5-46f9-9fec-fe7c59a5003c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:15:10.383501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.383480    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:15:10.383501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.383499    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:15:10.383676 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.383512    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:15:10.383676 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.383527    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:15:10.383676 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.383540    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:15:10.385070 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.385045    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8bca294c-83b5-46f9-9fec-fe7c59a5003c" (UID: "8bca294c-83b5-46f9-9fec-fe7c59a5003c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:15:10.385189 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.385118    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-kube-api-access-2ffp5" (OuterVolumeSpecName: "kube-api-access-2ffp5") pod "8bca294c-83b5-46f9-9fec-fe7c59a5003c" (UID: "8bca294c-83b5-46f9-9fec-fe7c59a5003c"). InnerVolumeSpecName "kube-api-access-2ffp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:15:10.385238 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.385184    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8bca294c-83b5-46f9-9fec-fe7c59a5003c" (UID: "8bca294c-83b5-46f9-9fec-fe7c59a5003c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:15:10.385313 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.385297    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-token" (OuterVolumeSpecName: "istio-token") pod "8bca294c-83b5-46f9-9fec-fe7c59a5003c" (UID: "8bca294c-83b5-46f9-9fec-fe7c59a5003c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:15:10.484799 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.484756    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:15:10.484799 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.484795    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:15:10.484799 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.484806    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ffp5\" (UniqueName: \"kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-kube-api-access-2ffp5\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:15:10.484799 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:10.484816    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bca294c-83b5-46f9-9fec-fe7c59a5003c-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:15:11.079344 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.079310    2559 generic.go:356] "Generic (PLEG): container finished" podID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerID="ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128" exitCode=0
Sep 11 01:15:11.079510 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.079366    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"
Sep 11 01:15:11.079510 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.079387    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" event={"ID":"8bca294c-83b5-46f9-9fec-fe7c59a5003c","Type":"ContainerDied","Data":"ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128"}
Sep 11 01:15:11.079510 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.079437    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr" event={"ID":"8bca294c-83b5-46f9-9fec-fe7c59a5003c","Type":"ContainerDied","Data":"76600dbd6983c6b36f62053de5a49d3a35dcc85f9feda922b104210962924177"}
Sep 11 01:15:11.079510 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.079459    2559 scope.go:117] "RemoveContainer" containerID="ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128"
Sep 11 01:15:11.090269 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.090086    2559 scope.go:117] "RemoveContainer" containerID="b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085"
Sep 11 01:15:11.099807 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.099792    2559 scope.go:117] "RemoveContainer" containerID="ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128"
Sep 11 01:15:11.100102 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:15:11.100051    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128\": container with ID starting with ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128 not found: ID does not exist" containerID="ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128"
Sep 11 01:15:11.100102 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.100079    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128"} err="failed to get container status \"ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128\": rpc error: code = NotFound desc = could not find container \"ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128\": container with ID starting with ef7f15edb21dee3c34ce1e1fd078d1c1e34a282e25c83761064f5b188fa29128 not found: ID does not exist"
Sep 11 01:15:11.100102 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.100100    2559 scope.go:117] "RemoveContainer" containerID="b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085"
Sep 11 01:15:11.100426 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:15:11.100401    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085\": container with ID starting with b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085 not found: ID does not exist" containerID="b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085"
Sep 11 01:15:11.100514 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.100436    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085"} err="failed to get container status \"b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085\": rpc error: code = NotFound desc = could not find container \"b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085\": container with ID starting with b5114739808934ecf2741f76dde52503d598de619bd6d096c0c26b280d763085 not found: ID does not exist"
Sep 11 01:15:11.111425 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.111395    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"]
Sep 11 01:15:11.111553 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:11.111431    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-2c1e69-openshift-default-5994fd4578-7wnbr"]
Sep 11 01:15:13.110544 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:15:13.110509    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" path="/var/lib/kubelet/pods/8bca294c-83b5-46f9-9fec-fe7c59a5003c/volumes"
Sep 11 01:16:28.448934 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.448889    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"]
Sep 11 01:16:28.449671 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449585    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy"
Sep 11 01:16:28.449671 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449621    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy"
Sep 11 01:16:28.449671 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449634    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy"
Sep 11 01:16:28.449671 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449641    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy"
Sep 11 01:16:28.449671 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449650    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy"
Sep 11 01:16:28.449671 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449659    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy"
Sep 11 01:16:28.450148 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449678    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy"
Sep 11 01:16:28.450148 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449688    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy"
Sep 11 01:16:28.450148 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449795    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy"
Sep 11 01:16:28.450148 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449808    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy"
Sep 11 01:16:28.450148 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449820    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ad3a4b7-6362-4859-88b4-8f94a14a6f83" containerName="istio-proxy"
Sep 11 01:16:28.450148 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.449831    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bca294c-83b5-46f9-9fec-fe7c59a5003c" containerName="istio-proxy"
Sep 11 01:16:28.452491 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.452470    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.455799 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.455781    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ef3c90aa-openshift-default-dockercfg-ghtl5\""
Sep 11 01:16:28.465911 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.465890    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"]
Sep 11 01:16:28.586276 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.586234    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66h6r\" (UniqueName: \"kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-kube-api-access-66h6r\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.586470 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.586291    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-envoy\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.586470 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.586323    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-credential-socket\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.586470 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.586375    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-certs\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.586470 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.586418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-token\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.586470 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.586444    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-socket\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.586758 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.586648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-data\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.586758 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.586710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-podinfo\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.586848 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.586758    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f05978c7-a8c8-44f3-a26e-41a07babf40a-istiod-ca-cert\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.687707 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.687656    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-certs\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.687903 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.687728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-token\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.687903 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.687757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-socket\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.687903 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.687839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-data\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.687903 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.687888    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-podinfo\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.688124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.687930    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f05978c7-a8c8-44f3-a26e-41a07babf40a-istiod-ca-cert\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.688124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.687998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66h6r\" (UniqueName: \"kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-kube-api-access-66h6r\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.688124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.688025    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-envoy\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.688124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.688063    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-credential-socket\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.688531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.688503    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-credential-socket\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.689422 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.689206    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-data\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.689528 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.689470    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-certs\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.690124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.689779    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-socket\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.690283 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.690250    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f05978c7-a8c8-44f3-a26e-41a07babf40a-istiod-ca-cert\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.692927 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.692866    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-podinfo\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.693042 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.692956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-envoy\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.700688 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.700590    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-token\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.700800 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.700685    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66h6r\" (UniqueName: \"kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-kube-api-access-66h6r\") pod \"e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") " pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:28.763684 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:28.763649    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:29.072839 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:16:29.072708    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05978c7_a8c8_44f3_a26e_41a07babf40a.slice/crio-f60bfcdd4b66a7bbf349f472926c0a8c4652049ebf4965600e5456e4cde77960 WatchSource:0}: Error finding container f60bfcdd4b66a7bbf349f472926c0a8c4652049ebf4965600e5456e4cde77960: Status 404 returned error can't find the container with id f60bfcdd4b66a7bbf349f472926c0a8c4652049ebf4965600e5456e4cde77960
Sep 11 01:16:29.075232 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:29.075194    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:16:29.075323 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:29.075269    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:16:29.075323 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:29.075297    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:16:29.075441 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:29.075421    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"]
Sep 11 01:16:29.563314 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:29.563272    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" event={"ID":"f05978c7-a8c8-44f3-a26e-41a07babf40a","Type":"ContainerStarted","Data":"9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0"}
Sep 11 01:16:29.563755 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:29.563330    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" event={"ID":"f05978c7-a8c8-44f3-a26e-41a07babf40a","Type":"ContainerStarted","Data":"f60bfcdd4b66a7bbf349f472926c0a8c4652049ebf4965600e5456e4cde77960"}
Sep 11 01:16:29.584841 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:29.584785    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" podStartSLOduration=1.584770577 podStartE2EDuration="1.584770577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:16:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:16:29.583649737 +0000 UTC m=+1109.098249923" watchObservedRunningTime="2026-09-11 01:16:29.584770577 +0000 UTC m=+1109.099370833"
Sep 11 01:16:29.764138 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:29.764107    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:30.769485 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:30.769458    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:31.575271 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:31.575237    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:31.576112 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:31.576089    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:16:35.947538 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:35.947501    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"]
Sep 11 01:16:35.950677 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:35.950656    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:35.953144 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:35.953124    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ef3c90aa-serving-cert\""
Sep 11 01:16:35.963751 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:35.963725    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"]
Sep 11 01:16:35.972690 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:35.972660    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmjln\" (UniqueName: \"kubernetes.io/projected/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-kube-api-access-gmjln\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:35.972838 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:35.972710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-openshift-service-ca\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:35.972925 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:35.972869    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-maas-api-tls\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:35.973009 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:35.972988    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-metrics-certs\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:36.074909 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:36.074858    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmjln\" (UniqueName: \"kubernetes.io/projected/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-kube-api-access-gmjln\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:36.075644 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:36.075616    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-openshift-service-ca\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:36.076227 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:36.076202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-maas-api-tls\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:36.076330 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:36.076319    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-metrics-certs\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:36.076832 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:16:36.076498    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-serving-cert: secret "maas-api-e2e-mt-ef3c90aa-serving-cert" not found
Sep 11 01:16:36.076832 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:16:36.076618    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-maas-api-tls podName:269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0 nodeName:}" failed. No retries permitted until 2026-09-11 01:16:36.576578415 +0000 UTC m=+1116.091178586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-maas-api-tls") pod "maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp" (UID: "269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0") : secret "maas-api-e2e-mt-ef3c90aa-serving-cert" not found
Sep 11 01:16:36.077451 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:36.077427    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-openshift-service-ca\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:36.079908 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:36.079883    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-metrics-certs\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:36.085321 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:36.085296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmjln\" (UniqueName: \"kubernetes.io/projected/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-kube-api-access-gmjln\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:36.583131 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:36.583086    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-maas-api-tls\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:36.585538 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:36.585513    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-maas-api-tls\") pod \"maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:36.867774 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:36.867684    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:37.150844 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:37.150818    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"]
Sep 11 01:16:37.151668 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:16:37.151632    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269a8a0f_51b7_4c7e_91b9_c7d1de6ecbe0.slice/crio-eefaf26209a2ef329da954ff3989d94015920556ea5955dd86e5f1ad0418ef85 WatchSource:0}: Error finding container eefaf26209a2ef329da954ff3989d94015920556ea5955dd86e5f1ad0418ef85: Status 404 returned error can't find the container with id eefaf26209a2ef329da954ff3989d94015920556ea5955dd86e5f1ad0418ef85
Sep 11 01:16:37.615031 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:37.614990    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp" event={"ID":"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0","Type":"ContainerStarted","Data":"a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006"}
Sep 11 01:16:37.615177 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:37.615042    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp" event={"ID":"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0","Type":"ContainerStarted","Data":"eefaf26209a2ef329da954ff3989d94015920556ea5955dd86e5f1ad0418ef85"}
Sep 11 01:16:37.615177 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:37.615083    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:16:37.632699 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:37.632648    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp" podStartSLOduration=2.266139483 podStartE2EDuration="2.632632185s" totalImagesPullingTime="366.492702ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:16:37.631540133 +0000 UTC m=+1117.146140344" watchObservedRunningTime="2026-09-11 01:16:37.632632185 +0000 UTC m=+1117.147232367"
Sep 11 01:16:43.630056 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:16:43.630025    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:17:01.713845 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:01.713812    2559 patch_prober.go:28] interesting pod/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": read tcp 10.133.0.2:47868->10.133.0.66:15021: read: connection reset by peer" start-of-body=
Sep 11 01:17:01.714179 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:01.713887    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": read tcp 10.133.0.2:47868->10.133.0.66:15021: read: connection reset by peer"
Sep 11 01:17:01.770805 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:01.770777    2559 generic.go:356] "Generic (PLEG): container finished" podID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerID="9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0" exitCode=0
Sep 11 01:17:01.770934 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:01.770850    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" event={"ID":"f05978c7-a8c8-44f3-a26e-41a07babf40a","Type":"ContainerDied","Data":"9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0"}
Sep 11 01:17:01.771273 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:01.771258    2559 scope.go:117] "RemoveContainer" containerID="9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0"
Sep 11 01:17:01.772392 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:01.772366    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:17:01.772447 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:01.772428    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:17:01.772484 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:01.772458    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:17:02.778681 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:02.778640    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" event={"ID":"f05978c7-a8c8-44f3-a26e-41a07babf40a","Type":"ContainerStarted","Data":"bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351"}
Sep 11 01:17:03.764477 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:03.764435    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:17:03.764806 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:03.764779    2559 patch_prober.go:28] interesting pod/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 11 01:17:03.764868 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:03.764844    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 11 01:17:04.765129 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:04.765094    2559 patch_prober.go:28] interesting pod/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 11 01:17:04.765537 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:04.765152    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 11 01:17:05.765055 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:05.765016    2559 patch_prober.go:28] interesting pod/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 11 01:17:05.765245 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:05.765071    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 11 01:17:06.764278 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:06.764240    2559 patch_prober.go:28] interesting pod/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 11 01:17:06.764523 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:06.764300    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 11 01:17:07.797247 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:07.797209    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:17:07.797739 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:07.797641    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:17:07.798481 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:07.798454    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:17:17.564721 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.564690    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"]
Sep 11 01:17:17.565150 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.564905    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp" podUID="269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0" containerName="maas-api" containerID="cri-o://a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006" gracePeriod=30
Sep 11 01:17:17.824776 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.824716    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:17:17.885378 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.884863    2559 generic.go:356] "Generic (PLEG): container finished" podID="269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0" containerID="a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006" exitCode=0
Sep 11 01:17:17.885378 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.885096    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp" event={"ID":"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0","Type":"ContainerDied","Data":"a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006"}
Sep 11 01:17:17.885378 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.885190    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp" event={"ID":"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0","Type":"ContainerDied","Data":"eefaf26209a2ef329da954ff3989d94015920556ea5955dd86e5f1ad0418ef85"}
Sep 11 01:17:17.885378 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.885242    2559 scope.go:117] "RemoveContainer" containerID="a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006"
Sep 11 01:17:17.885775 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.885480    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"
Sep 11 01:17:17.901680 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.901633    2559 scope.go:117] "RemoveContainer" containerID="a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006"
Sep 11 01:17:17.902057 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:17:17.902003    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006\": container with ID starting with a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006 not found: ID does not exist" containerID="a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006"
Sep 11 01:17:17.902238 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.902202    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006"} err="failed to get container status \"a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006\": rpc error: code = NotFound desc = could not find container \"a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006\": container with ID starting with a91c72a843046dd8409e7a2b0352e3e31638db5921bd949983dc0d625537b006 not found: ID does not exist"
Sep 11 01:17:17.970502 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.967244    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-metrics-certs\") pod \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") "
Sep 11 01:17:17.970502 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.967763    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmjln\" (UniqueName: \"kubernetes.io/projected/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-kube-api-access-gmjln\") pod \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") "
Sep 11 01:17:17.970502 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.968226    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-openshift-service-ca\") pod \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") "
Sep 11 01:17:17.970502 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.968586    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0" (UID: "269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:17:17.970502 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.968693    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-maas-api-tls\") pod \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\" (UID: \"269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0\") "
Sep 11 01:17:17.970502 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.969418    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:17.981344 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.981301    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0" (UID: "269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:17:17.982196 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.982149    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0" (UID: "269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:17:17.984103 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:17.984070    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-kube-api-access-gmjln" (OuterVolumeSpecName: "kube-api-access-gmjln") pod "269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0" (UID: "269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0"). InnerVolumeSpecName "kube-api-access-gmjln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:17:18.070179 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:18.070149    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:18.070179 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:18.070176    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmjln\" (UniqueName: \"kubernetes.io/projected/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-kube-api-access-gmjln\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:18.070179 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:18.070186    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:18.218434 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:18.218406    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"]
Sep 11 01:17:18.228134 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:18.228107    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ef3c90aa-55f7cfd4cc-jqdkp"]
Sep 11 01:17:19.111072 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:19.111038    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0" path="/var/lib/kubelet/pods/269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0/volumes"
Sep 11 01:17:24.326336 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:24.326254    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"]
Sep 11 01:17:24.326703 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:24.326502    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy" containerID="cri-o://bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351" gracePeriod=30
Sep 11 01:17:29.588881 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.588843    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:17:29.689943 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.689856    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-socket\") pod \"f05978c7-a8c8-44f3-a26e-41a07babf40a\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") "
Sep 11 01:17:29.689943 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.689900    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-podinfo\") pod \"f05978c7-a8c8-44f3-a26e-41a07babf40a\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") "
Sep 11 01:17:29.690173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.689949    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-certs\") pod \"f05978c7-a8c8-44f3-a26e-41a07babf40a\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") "
Sep 11 01:17:29.690173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.689964    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f05978c7-a8c8-44f3-a26e-41a07babf40a-istiod-ca-cert\") pod \"f05978c7-a8c8-44f3-a26e-41a07babf40a\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") "
Sep 11 01:17:29.690173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.689984    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-credential-socket\") pod \"f05978c7-a8c8-44f3-a26e-41a07babf40a\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") "
Sep 11 01:17:29.690173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690032    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66h6r\" (UniqueName: \"kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-kube-api-access-66h6r\") pod \"f05978c7-a8c8-44f3-a26e-41a07babf40a\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") "
Sep 11 01:17:29.690173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690061    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-envoy\") pod \"f05978c7-a8c8-44f3-a26e-41a07babf40a\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") "
Sep 11 01:17:29.690427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690172    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f05978c7-a8c8-44f3-a26e-41a07babf40a" (UID: "f05978c7-a8c8-44f3-a26e-41a07babf40a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:17:29.690427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690196    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-data\") pod \"f05978c7-a8c8-44f3-a26e-41a07babf40a\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") "
Sep 11 01:17:29.690427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690267    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f05978c7-a8c8-44f3-a26e-41a07babf40a" (UID: "f05978c7-a8c8-44f3-a26e-41a07babf40a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:17:29.690427 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690273    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-token\") pod \"f05978c7-a8c8-44f3-a26e-41a07babf40a\" (UID: \"f05978c7-a8c8-44f3-a26e-41a07babf40a\") "
Sep 11 01:17:29.690672 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690653    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:29.690734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690672    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:29.690734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690676    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f05978c7-a8c8-44f3-a26e-41a07babf40a" (UID: "f05978c7-a8c8-44f3-a26e-41a07babf40a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:17:29.690734 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690693    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-data" (OuterVolumeSpecName: "istio-data") pod "f05978c7-a8c8-44f3-a26e-41a07babf40a" (UID: "f05978c7-a8c8-44f3-a26e-41a07babf40a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:17:29.690887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.690764    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f05978c7-a8c8-44f3-a26e-41a07babf40a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f05978c7-a8c8-44f3-a26e-41a07babf40a" (UID: "f05978c7-a8c8-44f3-a26e-41a07babf40a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:17:29.692289 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.692265    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f05978c7-a8c8-44f3-a26e-41a07babf40a" (UID: "f05978c7-a8c8-44f3-a26e-41a07babf40a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:17:29.692387 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.692346    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-kube-api-access-66h6r" (OuterVolumeSpecName: "kube-api-access-66h6r") pod "f05978c7-a8c8-44f3-a26e-41a07babf40a" (UID: "f05978c7-a8c8-44f3-a26e-41a07babf40a"). InnerVolumeSpecName "kube-api-access-66h6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:17:29.692616 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.692581    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f05978c7-a8c8-44f3-a26e-41a07babf40a" (UID: "f05978c7-a8c8-44f3-a26e-41a07babf40a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:17:29.692744 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.692727    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-token" (OuterVolumeSpecName: "istio-token") pod "f05978c7-a8c8-44f3-a26e-41a07babf40a" (UID: "f05978c7-a8c8-44f3-a26e-41a07babf40a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:17:29.792063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.792031    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:29.792063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.792058    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f05978c7-a8c8-44f3-a26e-41a07babf40a-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:29.792063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.792067    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:29.792297 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.792079    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66h6r\" (UniqueName: \"kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-kube-api-access-66h6r\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:29.792297 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.792088    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:29.792297 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.792098    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:29.792297 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.792105    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f05978c7-a8c8-44f3-a26e-41a07babf40a-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:17:29.962357 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.962280    2559 generic.go:356] "Generic (PLEG): container finished" podID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerID="bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351" exitCode=0
Sep 11 01:17:29.962357 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.962322    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"
Sep 11 01:17:29.962528 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.962363    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" event={"ID":"f05978c7-a8c8-44f3-a26e-41a07babf40a","Type":"ContainerDied","Data":"bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351"}
Sep 11 01:17:29.962528 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.962416    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg" event={"ID":"f05978c7-a8c8-44f3-a26e-41a07babf40a","Type":"ContainerDied","Data":"f60bfcdd4b66a7bbf349f472926c0a8c4652049ebf4965600e5456e4cde77960"}
Sep 11 01:17:29.962528 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.962438    2559 scope.go:117] "RemoveContainer" containerID="bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351"
Sep 11 01:17:29.972492 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.972475    2559 scope.go:117] "RemoveContainer" containerID="9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0"
Sep 11 01:17:29.981206 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.981189    2559 scope.go:117] "RemoveContainer" containerID="bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351"
Sep 11 01:17:29.981584 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:17:29.981558    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351\": container with ID starting with bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351 not found: ID does not exist" containerID="bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351"
Sep 11 01:17:29.981826 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.981593    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351"} err="failed to get container status \"bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351\": rpc error: code = NotFound desc = could not find container \"bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351\": container with ID starting with bdd17bd372398fb3519ee20f8b44a34982f8e1951e8261123606a21eb16ea351 not found: ID does not exist"
Sep 11 01:17:29.981826 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.981635    2559 scope.go:117] "RemoveContainer" containerID="9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0"
Sep 11 01:17:29.982217 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:17:29.981958    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0\": container with ID starting with 9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0 not found: ID does not exist" containerID="9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0"
Sep 11 01:17:29.982217 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.981990    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0"} err="failed to get container status \"9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0\": rpc error: code = NotFound desc = could not find container \"9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0\": container with ID starting with 9266f53f5211b9456b24ba984aa024a39371200c5810bf5dc3e71db16b58d6d0 not found: ID does not exist"
Sep 11 01:17:29.985348 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.985298    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"]
Sep 11 01:17:29.988690 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:29.988664    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ef3c90aa-openshift-default-7b99494568-djzbg"]
Sep 11 01:17:31.112296 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:31.112264    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" path="/var/lib/kubelet/pods/f05978c7-a8c8-44f3-a26e-41a07babf40a/volumes"
Sep 11 01:17:37.784619 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.784570    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"]
Sep 11 01:17:37.785280 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.785259    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0" containerName="maas-api"
Sep 11 01:17:37.785372 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.785284    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0" containerName="maas-api"
Sep 11 01:17:37.785372 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.785328    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy"
Sep 11 01:17:37.785372 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.785336    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy"
Sep 11 01:17:37.785372 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.785346    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy"
Sep 11 01:17:37.785372 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.785353    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy"
Sep 11 01:17:37.785646 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.785456    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy"
Sep 11 01:17:37.785646 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.785473    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="269a8a0f-51b7-4c7e-91b9-c7d1de6ecbe0" containerName="maas-api"
Sep 11 01:17:37.787859 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.787836    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.790793 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.790774    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-76bb56-openshift-default-dockercfg-d8xwp\""
Sep 11 01:17:37.799326 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.799305    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"]
Sep 11 01:17:37.876091 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.875943    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-token\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.876091 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.876021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-data\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.876364 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.876224    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-socket\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.876553 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.876370    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-certs\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.876937 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.876599    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-envoy\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.876937 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.876684    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgjz5\" (UniqueName: \"kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-kube-api-access-mgjz5\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.876937 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.876718    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-credential-socket\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.876937 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.876766    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-podinfo\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.876937 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.876803    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istiod-ca-cert\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.977428 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.977391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-envoy\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.977428 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.977431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgjz5\" (UniqueName: \"kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-kube-api-access-mgjz5\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.977714 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.977452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-credential-socket\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.977714 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.977469    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-podinfo\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.977714 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.977499    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istiod-ca-cert\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.977714 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.977553    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-token\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.977714 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.977591    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-data\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.977714 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.977669    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-socket\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.978052 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.977747    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-certs\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.978052 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.977918    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-credential-socket\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.978159 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.978052    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-socket\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.978383 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.978351    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istiod-ca-cert\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.978514 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.978405    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-certs\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.978514 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.978498    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-data\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.979858 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.979836    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-envoy\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.980133 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.980114    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-podinfo\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.986757 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.986732    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgjz5\" (UniqueName: \"kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-kube-api-access-mgjz5\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:37.987380 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:37.987358    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-token\") pod \"e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") " pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:38.100110 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:38.100030    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:38.377002 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:38.376969    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"]
Sep 11 01:17:38.377344 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:17:38.377319    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0383f8_fbe2_40c6_aeb4_bae3da376212.slice/crio-e0f3c7886a235f21ffe6e9e919ef21cd165022a5539da1711e02b2ccdf1201f9 WatchSource:0}: Error finding container e0f3c7886a235f21ffe6e9e919ef21cd165022a5539da1711e02b2ccdf1201f9: Status 404 returned error can't find the container with id e0f3c7886a235f21ffe6e9e919ef21cd165022a5539da1711e02b2ccdf1201f9
Sep 11 01:17:38.379878 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:38.379837    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:17:38.379945 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:38.379926    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:17:38.380005 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:38.379971    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:17:39.021317 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:39.021282    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" event={"ID":"fc0383f8-fbe2-40c6-aeb4-bae3da376212","Type":"ContainerStarted","Data":"d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2"}
Sep 11 01:17:39.021806 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:39.021331    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" event={"ID":"fc0383f8-fbe2-40c6-aeb4-bae3da376212","Type":"ContainerStarted","Data":"e0f3c7886a235f21ffe6e9e919ef21cd165022a5539da1711e02b2ccdf1201f9"}
Sep 11 01:17:39.045930 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:39.045877    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" podStartSLOduration=2.045862313 podStartE2EDuration="2.045862313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:17:39.04389989 +0000 UTC m=+1178.558500074" watchObservedRunningTime="2026-09-11 01:17:39.045862313 +0000 UTC m=+1178.560462507"
Sep 11 01:17:39.100481 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:39.100449    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:39.111114 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:39.111089    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:40.027334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:40.027308    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:40.028406 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:40.028382    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:17:42.213242 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.213202    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"]
Sep 11 01:17:42.214130 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.214102    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f05978c7-a8c8-44f3-a26e-41a07babf40a" containerName="istio-proxy"
Sep 11 01:17:42.217039 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.217018    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.219872 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.219849    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-76bb56-serving-cert\""
Sep 11 01:17:42.231241 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.231214    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"]
Sep 11 01:17:42.323657 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.323592    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.323859 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.323683    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.323859 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.323796    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjlqd\" (UniqueName: \"kubernetes.io/projected/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-kube-api-access-mjlqd\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.323859 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.323856    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.425510 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.425472    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.425732 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.425514    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.425732 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.425582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjlqd\" (UniqueName: \"kubernetes.io/projected/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-kube-api-access-mjlqd\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.425732 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.425638    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.425732 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:17:42.425648    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-serving-cert: secret "maas-api-e2e-shared-a-w2-76bb56-serving-cert" not found
Sep 11 01:17:42.425732 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:17:42.425733    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-maas-api-tls podName:5d63e28a-7e91-4aa5-98b2-f77e49053c2d nodeName:}" failed. No retries permitted until 2026-09-11 01:17:42.925709569 +0000 UTC m=+1182.440309748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-maas-api-tls") pod "maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92" (UID: "5d63e28a-7e91-4aa5-98b2-f77e49053c2d") : secret "maas-api-e2e-shared-a-w2-76bb56-serving-cert" not found
Sep 11 01:17:42.426295 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.426270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.428159 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.428137    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.436780 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.436759    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjlqd\" (UniqueName: \"kubernetes.io/projected/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-kube-api-access-mjlqd\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.931065 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.931033    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:42.933576 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:42.933551    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:43.130685 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:43.130648    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:43.404015 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:43.403985    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"]
Sep 11 01:17:43.404503 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:17:43.404481    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d63e28a_7e91_4aa5_98b2_f77e49053c2d.slice/crio-dc19fb313829c7e285e424e59ed8144157697a772402c2fddec749954287fb14 WatchSource:0}: Error finding container dc19fb313829c7e285e424e59ed8144157697a772402c2fddec749954287fb14: Status 404 returned error can't find the container with id dc19fb313829c7e285e424e59ed8144157697a772402c2fddec749954287fb14
Sep 11 01:17:44.058351 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:44.058317    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92" event={"ID":"5d63e28a-7e91-4aa5-98b2-f77e49053c2d","Type":"ContainerStarted","Data":"b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55"}
Sep 11 01:17:44.058530 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:44.058377    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:17:44.058530 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:44.058394    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92" event={"ID":"5d63e28a-7e91-4aa5-98b2-f77e49053c2d","Type":"ContainerStarted","Data":"dc19fb313829c7e285e424e59ed8144157697a772402c2fddec749954287fb14"}
Sep 11 01:17:44.078791 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:44.078742    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92" podStartSLOduration=1.5701929369999998 podStartE2EDuration="2.078727107s" totalImagesPullingTime="508.53417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:17:44.075513353 +0000 UTC m=+1183.590113536" watchObservedRunningTime="2026-09-11 01:17:44.078727107 +0000 UTC m=+1183.593327293"
Sep 11 01:17:50.070680 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:17:50.070652    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:18:01.166663 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:01.166630    2559 generic.go:356] "Generic (PLEG): container finished" podID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerID="d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2" exitCode=0
Sep 11 01:18:01.167046 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:01.166676    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" event={"ID":"fc0383f8-fbe2-40c6-aeb4-bae3da376212","Type":"ContainerDied","Data":"d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2"}
Sep 11 01:18:01.167089 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:01.167053    2559 scope.go:117] "RemoveContainer" containerID="d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2"
Sep 11 01:18:01.168158 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:01.168123    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:18:01.168285 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:01.168203    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:18:01.168285 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:01.168250    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:18:02.174924 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:02.174889    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" event={"ID":"fc0383f8-fbe2-40c6-aeb4-bae3da376212","Type":"ContainerStarted","Data":"19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419"}
Sep 11 01:18:03.100788 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:03.100750    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:18:03.101001 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:03.100969    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 11 01:18:03.101077 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:03.101029    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 11 01:18:04.100654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:04.100617    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 11 01:18:04.101038 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:04.100677    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 11 01:18:05.081646 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.079035    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"]
Sep 11 01:18:05.081646 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.079357    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j" podUID="091d0047-2df1-4ea3-a8a3-8635bc79ca61" containerName="maas-api" containerID="cri-o://3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc" gracePeriod=30
Sep 11 01:18:05.101468 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.101438    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 11 01:18:05.102027 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.101990    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 11 01:18:05.374780 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.374755    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:18:05.487709 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.487672    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-metrics-certs\") pod \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") "
Sep 11 01:18:05.487899 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.487758    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/091d0047-2df1-4ea3-a8a3-8635bc79ca61-openshift-service-ca\") pod \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") "
Sep 11 01:18:05.487899 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.487803    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v22q\" (UniqueName: \"kubernetes.io/projected/091d0047-2df1-4ea3-a8a3-8635bc79ca61-kube-api-access-4v22q\") pod \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") "
Sep 11 01:18:05.487899 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.487866    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-maas-api-tls\") pod \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\" (UID: \"091d0047-2df1-4ea3-a8a3-8635bc79ca61\") "
Sep 11 01:18:05.488132 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.488101    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/091d0047-2df1-4ea3-a8a3-8635bc79ca61-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "091d0047-2df1-4ea3-a8a3-8635bc79ca61" (UID: "091d0047-2df1-4ea3-a8a3-8635bc79ca61"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:18:05.488316 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.488297    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/091d0047-2df1-4ea3-a8a3-8635bc79ca61-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:05.489906 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.489874    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "091d0047-2df1-4ea3-a8a3-8635bc79ca61" (UID: "091d0047-2df1-4ea3-a8a3-8635bc79ca61"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:18:05.490288 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.490266    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "091d0047-2df1-4ea3-a8a3-8635bc79ca61" (UID: "091d0047-2df1-4ea3-a8a3-8635bc79ca61"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:18:05.490384 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.490309    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091d0047-2df1-4ea3-a8a3-8635bc79ca61-kube-api-access-4v22q" (OuterVolumeSpecName: "kube-api-access-4v22q") pod "091d0047-2df1-4ea3-a8a3-8635bc79ca61" (UID: "091d0047-2df1-4ea3-a8a3-8635bc79ca61"). InnerVolumeSpecName "kube-api-access-4v22q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:18:05.589808 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.589712    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v22q\" (UniqueName: \"kubernetes.io/projected/091d0047-2df1-4ea3-a8a3-8635bc79ca61-kube-api-access-4v22q\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:05.589808 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.589746    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:05.589808 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:05.589759    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/091d0047-2df1-4ea3-a8a3-8635bc79ca61-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:06.101337 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.101298    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 11 01:18:06.101519 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.101377    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 11 01:18:06.205582 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.205543    2559 generic.go:356] "Generic (PLEG): container finished" podID="091d0047-2df1-4ea3-a8a3-8635bc79ca61" containerID="3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc" exitCode=0
Sep 11 01:18:06.205779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.205599    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"
Sep 11 01:18:06.205779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.205630    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j" event={"ID":"091d0047-2df1-4ea3-a8a3-8635bc79ca61","Type":"ContainerDied","Data":"3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc"}
Sep 11 01:18:06.205779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.205688    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j" event={"ID":"091d0047-2df1-4ea3-a8a3-8635bc79ca61","Type":"ContainerDied","Data":"41577ff931e2eb9823a48b65c0de2ce4543bb81ca6056efc0a6935cd67a72b8d"}
Sep 11 01:18:06.205779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.205714    2559 scope.go:117] "RemoveContainer" containerID="3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc"
Sep 11 01:18:06.219725 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.219701    2559 scope.go:117] "RemoveContainer" containerID="3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc"
Sep 11 01:18:06.219978 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:18:06.219960    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc\": container with ID starting with 3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc not found: ID does not exist" containerID="3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc"
Sep 11 01:18:06.220028 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.219989    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc"} err="failed to get container status \"3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc\": rpc error: code = NotFound desc = could not find container \"3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc\": container with ID starting with 3520e977a673b434f8b70266ad8d1c19e0c4e65570a816288c3f4bab341946dc not found: ID does not exist"
Sep 11 01:18:06.229812 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.229786    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"]
Sep 11 01:18:06.233923 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:06.233900    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cd176e-6c954cc7d8-hs59j"]
Sep 11 01:18:07.101374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:07.101328    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 11 01:18:07.101566 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:07.101406    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 11 01:18:07.110912 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:07.110882    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091d0047-2df1-4ea3-a8a3-8635bc79ca61" path="/var/lib/kubelet/pods/091d0047-2df1-4ea3-a8a3-8635bc79ca61/volumes"
Sep 11 01:18:08.104578 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:08.104548    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:18:08.104965 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:08.104923    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:18:08.105667 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:08.105649    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:18:12.655617 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:12.655575    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"]
Sep 11 01:18:12.656035 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:12.655826    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy" containerID="cri-o://0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3" gracePeriod=30
Sep 11 01:18:16.181187 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:16.181147    2559 patch_prober.go:28] interesting pod/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 01:18:16.181548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:16.181210    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 01:18:17.911019 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:17.910994    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:18:18.022849 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.022820    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-socket\") pod \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") "
Sep 11 01:18:18.023045 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.022885    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-credential-socket\") pod \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") "
Sep 11 01:18:18.023045 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.022912    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-envoy\") pod \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") "
Sep 11 01:18:18.023045 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.022962    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-podinfo\") pod \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") "
Sep 11 01:18:18.023045 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023018    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvjml\" (UniqueName: \"kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-kube-api-access-cvjml\") pod \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") "
Sep 11 01:18:18.023264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023052    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istiod-ca-cert\") pod \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") "
Sep 11 01:18:18.023264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023106    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" (UID: "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:18.023264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023214    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" (UID: "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:18.023434 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023259    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-data\") pod \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") "
Sep 11 01:18:18.023434 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023310    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-token\") pod \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") "
Sep 11 01:18:18.023434 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023402    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-certs\") pod \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\" (UID: \"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e\") "
Sep 11 01:18:18.023576 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023488    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-data" (OuterVolumeSpecName: "istio-data") pod "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" (UID: "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:18.023576 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023423    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" (UID: "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:18:18.023768 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023743    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" (UID: "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:18.024036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023894    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:18.024036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023913    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:18.024036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023928    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:18.024036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023942    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:18.024036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.023955    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:18.025227 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.025201    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" (UID: "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:18:18.025227 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.025204    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-kube-api-access-cvjml" (OuterVolumeSpecName: "kube-api-access-cvjml") pod "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" (UID: "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e"). InnerVolumeSpecName "kube-api-access-cvjml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:18:18.025491 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.025471    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" (UID: "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:18:18.025591 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.025575    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-token" (OuterVolumeSpecName: "istio-token") pod "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" (UID: "0cfdb29b-d87a-41c4-b7ca-9c3525724d1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:18:18.125492 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.125444    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:18.125492 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.125486    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:18.125492 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.125495    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvjml\" (UniqueName: \"kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-kube-api-access-cvjml\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:18.125492 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.125504    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:18:18.285079 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.284997    2559 generic.go:356] "Generic (PLEG): container finished" podID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerID="0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3" exitCode=0
Sep 11 01:18:18.285079 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.285040    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" event={"ID":"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e","Type":"ContainerDied","Data":"0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3"}
Sep 11 01:18:18.285079 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.285045    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"
Sep 11 01:18:18.285079 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.285081    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj" event={"ID":"0cfdb29b-d87a-41c4-b7ca-9c3525724d1e","Type":"ContainerDied","Data":"989e6e5516a35e9c9ff3e6e68090a2de0b32b8f8d1b44b9f099e81bf5c28c959"}
Sep 11 01:18:18.285376 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.285102    2559 scope.go:117] "RemoveContainer" containerID="0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3"
Sep 11 01:18:18.296254 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.296237    2559 scope.go:117] "RemoveContainer" containerID="55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802"
Sep 11 01:18:18.305035 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.305017    2559 scope.go:117] "RemoveContainer" containerID="0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3"
Sep 11 01:18:18.305306 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:18:18.305280    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3\": container with ID starting with 0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3 not found: ID does not exist" containerID="0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3"
Sep 11 01:18:18.305408 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.305313    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3"} err="failed to get container status \"0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3\": rpc error: code = NotFound desc = could not find container \"0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3\": container with ID starting with 0ab5371348af67cf1e63fe9a5a3b711342f03746cf7e857e689123b51a643aa3 not found: ID does not exist"
Sep 11 01:18:18.305408 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.305335    2559 scope.go:117] "RemoveContainer" containerID="55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802"
Sep 11 01:18:18.305630 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:18:18.305596    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802\": container with ID starting with 55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802 not found: ID does not exist" containerID="55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802"
Sep 11 01:18:18.305720 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.305639    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802"} err="failed to get container status \"55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802\": rpc error: code = NotFound desc = could not find container \"55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802\": container with ID starting with 55378c5796eea55cb5b084840bf5c19a54a72aca20a2d05c4fd4d74afe707802 not found: ID does not exist"
Sep 11 01:18:18.309192 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.309170    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"]
Sep 11 01:18:18.315111 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:18.315088    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-cd176e-openshift-default-7fdc756776-m5cdj"]
Sep 11 01:18:19.110919 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:19.110881    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" path="/var/lib/kubelet/pods/0cfdb29b-d87a-41c4-b7ca-9c3525724d1e/volumes"
Sep 11 01:18:31.804398 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.804356    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"]
Sep 11 01:18:31.805597 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.805569    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091d0047-2df1-4ea3-a8a3-8635bc79ca61" containerName="maas-api"
Sep 11 01:18:31.805597 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.805599    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="091d0047-2df1-4ea3-a8a3-8635bc79ca61" containerName="maas-api"
Sep 11 01:18:31.805802 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.805644    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy"
Sep 11 01:18:31.805802 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.805654    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy"
Sep 11 01:18:31.805802 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.805673    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy"
Sep 11 01:18:31.805802 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.805683    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy"
Sep 11 01:18:31.806004 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.805935    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="091d0047-2df1-4ea3-a8a3-8635bc79ca61" containerName="maas-api"
Sep 11 01:18:31.806004 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.805964    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy"
Sep 11 01:18:31.806004 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.805975    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cfdb29b-d87a-41c4-b7ca-9c3525724d1e" containerName="istio-proxy"
Sep 11 01:18:31.820010 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.819810    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:31.822767 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.822729    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"]
Sep 11 01:18:31.822913 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.822847    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-4ec6e9-openshift-default-dockercfg-kdksz\""
Sep 11 01:18:31.966776 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.966738    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istiod-ca-cert\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:31.966776 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.966779    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-certs\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:31.966999 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.966871    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-podinfo\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:31.966999 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.966909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-envoy\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:31.966999 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.966967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-token\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:31.967114 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.967012    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-data\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:31.967114 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.967041    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-socket\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:31.967114 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.967099    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-credential-socket\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:31.967224 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:31.967122    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svstp\" (UniqueName: \"kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-kube-api-access-svstp\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.067897 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.067810    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-credential-socket\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.067897 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.067863    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svstp\" (UniqueName: \"kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-kube-api-access-svstp\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.067930    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istiod-ca-cert\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.067960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-certs\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.068046    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-podinfo\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068250 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.068097    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-envoy\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068250 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.068137    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-token\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068250 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.068180    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-data\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068250 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.068204    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-credential-socket\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068250 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.068222    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-socket\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068512 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.068371    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-certs\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068639 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.068594    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-socket\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.068639 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.068629    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-data\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.069059 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.069031    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istiod-ca-cert\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.070446 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.070425    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-envoy\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.070779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.070761    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-podinfo\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.075929 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.075893    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-token\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.076158 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.076136    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svstp\" (UniqueName: \"kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-kube-api-access-svstp\") pod \"e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") " pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.132575 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.132540    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:32.412715 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.412685    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"]
Sep 11 01:18:32.413599 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:18:32.413562    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ecb1e06_a25f_4093_b64b_5d5b41638b24.slice/crio-ebea5d453e2df38eab902c14d0878f95abaad09b62c1b8a5649e04002d7ecf85 WatchSource:0}: Error finding container ebea5d453e2df38eab902c14d0878f95abaad09b62c1b8a5649e04002d7ecf85: Status 404 returned error can't find the container with id ebea5d453e2df38eab902c14d0878f95abaad09b62c1b8a5649e04002d7ecf85
Sep 11 01:18:32.415999 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.415966    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:18:32.416080 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.416032    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:18:32.416080 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:32.416058    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:18:33.385397 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:33.385366    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s" event={"ID":"5ecb1e06-a25f-4093-b64b-5d5b41638b24","Type":"ContainerStarted","Data":"060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9"}
Sep 11 01:18:33.385807 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:33.385419    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s" event={"ID":"5ecb1e06-a25f-4093-b64b-5d5b41638b24","Type":"ContainerStarted","Data":"ebea5d453e2df38eab902c14d0878f95abaad09b62c1b8a5649e04002d7ecf85"}
Sep 11 01:18:33.407119 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:33.407073    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s" podStartSLOduration=2.407056918 podStartE2EDuration="2.407056918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:18:33.404399827 +0000 UTC m=+1232.919000011" watchObservedRunningTime="2026-09-11 01:18:33.407056918 +0000 UTC m=+1232.921657100"
Sep 11 01:18:34.132776 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:34.132741    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:34.138095 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:34.138065    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:34.392021 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:34.391930    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:34.392893 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:34.392876    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:18:49.500479 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.500435    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"]
Sep 11 01:18:49.505506 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.505480    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.508040 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.508007    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6f69f6e6-serving-cert\""
Sep 11 01:18:49.515551 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.515519    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"]
Sep 11 01:18:49.547777 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.547737    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-openshift-service-ca\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.547948 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.547800    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45grw\" (UniqueName: \"kubernetes.io/projected/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-kube-api-access-45grw\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.547948 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.547879    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-maas-api-tls\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.548064 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.547997    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-metrics-certs\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.648717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.648675    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-metrics-certs\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.648929 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.648749    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-openshift-service-ca\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.648929 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.648815    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45grw\" (UniqueName: \"kubernetes.io/projected/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-kube-api-access-45grw\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.648929 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.648844    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-maas-api-tls\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.649112 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:18:49.648975    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-serving-cert: secret "maas-api-e2e-mt-6f69f6e6-serving-cert" not found
Sep 11 01:18:49.649112 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:18:49.649059    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-maas-api-tls podName:f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b nodeName:}" failed. No retries permitted until 2026-09-11 01:18:50.149037078 +0000 UTC m=+1249.663637255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-maas-api-tls") pod "maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls" (UID: "f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b") : secret "maas-api-e2e-mt-6f69f6e6-serving-cert" not found
Sep 11 01:18:49.649533 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.649507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-openshift-service-ca\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.651333 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.651310    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-metrics-certs\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:49.659159 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:49.659134    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45grw\" (UniqueName: \"kubernetes.io/projected/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-kube-api-access-45grw\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:50.153835 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:50.153801    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-maas-api-tls\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:50.156173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:50.156149    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-maas-api-tls\") pod \"maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:50.420828 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:50.420732    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:50.698683 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:50.695313    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"]
Sep 11 01:18:50.700052 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:50.700028    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 01:18:51.503313 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:51.503226    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls" event={"ID":"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b","Type":"ContainerStarted","Data":"473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332"}
Sep 11 01:18:51.503313 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:51.503280    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls" event={"ID":"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b","Type":"ContainerStarted","Data":"ae703ee818bdeb93734aa4c81402385dde31063b1fffa8cb89e6eba460a2eb9f"}
Sep 11 01:18:51.503313 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:51.503305    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:18:51.520912 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:51.520855    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls" podStartSLOduration=2.096073589 podStartE2EDuration="2.520834377s" totalImagesPullingTime="424.760788ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:18:51.520353976 +0000 UTC m=+1251.034954159" watchObservedRunningTime="2026-09-11 01:18:51.520834377 +0000 UTC m=+1251.035434559"
Sep 11 01:18:57.516917 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:18:57.516810    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:19:57.983641 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:57.983593    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"]
Sep 11 01:19:57.987948 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:57.987916    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:57.992540 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:57.992516    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f4de10\"/\"openshift-service-ca.crt\""
Sep 11 01:19:57.992812 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:57.992532    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f4de10\"/\"default-dockercfg-mg9vz\""
Sep 11 01:19:57.992935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:57.992587    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-f4de10\"/\"kube-root-ca.crt\""
Sep 11 01:19:57.992935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:57.992640    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-f4de10\"/\"unconfigured-w0-f4de10-kserve-self-signed-certs\""
Sep 11 01:19:57.998981 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:57.998955    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"]
Sep 11 01:19:58.087936 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.087896    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-model-cache\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.088114 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.087964    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k522h\" (UniqueName: \"kubernetes.io/projected/d5b40cc7-b454-45cd-9206-d6927dd9c632-kube-api-access-k522h\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.088114 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.088004    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-dshm\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.088209 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.088146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-tmp-dir\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.088209 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.088186    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5b40cc7-b454-45cd-9206-d6927dd9c632-tls-certs\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.088305 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.088218    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-home\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.189592 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.189558    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-tmp-dir\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.189592 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.189599    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5b40cc7-b454-45cd-9206-d6927dd9c632-tls-certs\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.189935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.189673    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-home\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.189935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.189715    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-model-cache\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.189935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.189775    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k522h\" (UniqueName: \"kubernetes.io/projected/d5b40cc7-b454-45cd-9206-d6927dd9c632-kube-api-access-k522h\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.189935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.189822    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-dshm\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.190151 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.190037    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-tmp-dir\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.190151 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.190084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-home\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.190151 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.190117    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-model-cache\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.191955 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.191931    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-dshm\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.192212 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.192194    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5b40cc7-b454-45cd-9206-d6927dd9c632-tls-certs\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.197872 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.197848    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k522h\" (UniqueName: \"kubernetes.io/projected/d5b40cc7-b454-45cd-9206-d6927dd9c632-kube-api-access-k522h\") pod \"unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") " pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.299291 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.299256    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:19:58.580810 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.580784    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"]
Sep 11 01:19:58.582296 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:19:58.582269    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5b40cc7_b454_45cd_9206_d6927dd9c632.slice/crio-68311fc2b96efc76066bcd881ded90afb189296a5c668c4b5ba739c15e95ffd3 WatchSource:0}: Error finding container 68311fc2b96efc76066bcd881ded90afb189296a5c668c4b5ba739c15e95ffd3: Status 404 returned error can't find the container with id 68311fc2b96efc76066bcd881ded90afb189296a5c668c4b5ba739c15e95ffd3
Sep 11 01:19:58.921847 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.921763    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg" event={"ID":"d5b40cc7-b454-45cd-9206-d6927dd9c632","Type":"ContainerStarted","Data":"09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122"}
Sep 11 01:19:58.921847 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.921801    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg" event={"ID":"d5b40cc7-b454-45cd-9206-d6927dd9c632","Type":"ContainerStarted","Data":"68311fc2b96efc76066bcd881ded90afb189296a5c668c4b5ba739c15e95ffd3"}
Sep 11 01:19:58.941237 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:19:58.941184    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg" podStartSLOduration=1.941166962 podStartE2EDuration="1.941166962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:19:58.938315511 +0000 UTC m=+1318.452915729" watchObservedRunningTime="2026-09-11 01:19:58.941166962 +0000 UTC m=+1318.455767146"
Sep 11 01:20:08.299531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:08.299492    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:20:08.299531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:08.299540    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:20:08.312372 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:08.312343    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:20:08.997302 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:08.997275    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:20:27.253678 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.253471    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"]
Sep 11 01:20:27.254855 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.254510    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg" podUID="74872bf5-d708-4f26-8527-57077dd9813b" containerName="maas-api" containerID="cri-o://fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913" gracePeriod=30
Sep 11 01:20:27.536001 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.535978    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:20:27.618225 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.618182    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-maas-api-tls\") pod \"74872bf5-d708-4f26-8527-57077dd9813b\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") "
Sep 11 01:20:27.618412 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.618339    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-metrics-certs\") pod \"74872bf5-d708-4f26-8527-57077dd9813b\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") "
Sep 11 01:20:27.618517 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.618481    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhhht\" (UniqueName: \"kubernetes.io/projected/74872bf5-d708-4f26-8527-57077dd9813b-kube-api-access-lhhht\") pod \"74872bf5-d708-4f26-8527-57077dd9813b\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") "
Sep 11 01:20:27.618798 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.618774    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74872bf5-d708-4f26-8527-57077dd9813b-openshift-service-ca\") pod \"74872bf5-d708-4f26-8527-57077dd9813b\" (UID: \"74872bf5-d708-4f26-8527-57077dd9813b\") "
Sep 11 01:20:27.619193 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.619139    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74872bf5-d708-4f26-8527-57077dd9813b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "74872bf5-d708-4f26-8527-57077dd9813b" (UID: "74872bf5-d708-4f26-8527-57077dd9813b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:20:27.619925 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.619903    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74872bf5-d708-4f26-8527-57077dd9813b-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:20:27.621049 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.621007    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "74872bf5-d708-4f26-8527-57077dd9813b" (UID: "74872bf5-d708-4f26-8527-57077dd9813b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:20:27.621551 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.621516    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "74872bf5-d708-4f26-8527-57077dd9813b" (UID: "74872bf5-d708-4f26-8527-57077dd9813b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:20:27.621670 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.621573    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74872bf5-d708-4f26-8527-57077dd9813b-kube-api-access-lhhht" (OuterVolumeSpecName: "kube-api-access-lhhht") pod "74872bf5-d708-4f26-8527-57077dd9813b" (UID: "74872bf5-d708-4f26-8527-57077dd9813b"). InnerVolumeSpecName "kube-api-access-lhhht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:20:27.721409 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.721375    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhhht\" (UniqueName: \"kubernetes.io/projected/74872bf5-d708-4f26-8527-57077dd9813b-kube-api-access-lhhht\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:20:27.721409 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.721405    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:20:27.721409 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:27.721414    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74872bf5-d708-4f26-8527-57077dd9813b-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:20:28.111724 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:28.111689    2559 generic.go:356] "Generic (PLEG): container finished" podID="74872bf5-d708-4f26-8527-57077dd9813b" containerID="fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913" exitCode=0
Sep 11 01:20:28.111907 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:28.111742    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"
Sep 11 01:20:28.111907 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:28.111756    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg" event={"ID":"74872bf5-d708-4f26-8527-57077dd9813b","Type":"ContainerDied","Data":"fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913"}
Sep 11 01:20:28.111907 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:28.111806    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg" event={"ID":"74872bf5-d708-4f26-8527-57077dd9813b","Type":"ContainerDied","Data":"07b498028663a3031941e62a46413965109a1eb51a583c1727aa7a3ccf5b0b5b"}
Sep 11 01:20:28.111907 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:28.111838    2559 scope.go:117] "RemoveContainer" containerID="fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913"
Sep 11 01:20:28.122246 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:28.122222    2559 scope.go:117] "RemoveContainer" containerID="fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913"
Sep 11 01:20:28.122540 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:20:28.122522    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913\": container with ID starting with fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913 not found: ID does not exist" containerID="fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913"
Sep 11 01:20:28.122589 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:28.122548    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913"} err="failed to get container status \"fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913\": rpc error: code = NotFound desc = could not find container \"fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913\": container with ID starting with fefffdb2441d428dc2ec6edfb29d4c22cbdad50fed837d7076bef6c183257913 not found: ID does not exist"
Sep 11 01:20:28.141417 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:28.141386    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"]
Sep 11 01:20:28.144224 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:28.144201    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e98d1e-568ff4c8b9-dsqtg"]
Sep 11 01:20:29.111538 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:29.111502    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74872bf5-d708-4f26-8527-57077dd9813b" path="/var/lib/kubelet/pods/74872bf5-d708-4f26-8527-57077dd9813b/volumes"
Sep 11 01:20:57.960315 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:57.960274    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"]
Sep 11 01:20:57.960917 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:20:57.960528    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s" podUID="5ecb1e06-a25f-4093-b64b-5d5b41638b24" containerName="istio-proxy" containerID="cri-o://060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9" gracePeriod=30
Sep 11 01:21:03.222195 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.222125    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:21:03.296933 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.296898    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istiod-ca-cert\") pod \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") "
Sep 11 01:21:03.297127 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.296946    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-podinfo\") pod \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") "
Sep 11 01:21:03.297127 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.296974    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-certs\") pod \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") "
Sep 11 01:21:03.297127 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.296996    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-token\") pod \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") "
Sep 11 01:21:03.297127 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297045    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-credential-socket\") pod \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") "
Sep 11 01:21:03.297127 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297101    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svstp\" (UniqueName: \"kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-kube-api-access-svstp\") pod \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") "
Sep 11 01:21:03.297403 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297168    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-data\") pod \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") "
Sep 11 01:21:03.297403 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297211    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-envoy\") pod \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") "
Sep 11 01:21:03.297403 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297295    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-socket\") pod \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\" (UID: \"5ecb1e06-a25f-4093-b64b-5d5b41638b24\") "
Sep 11 01:21:03.297568 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297448    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5ecb1e06-a25f-4093-b64b-5d5b41638b24" (UID: "5ecb1e06-a25f-4093-b64b-5d5b41638b24"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:21:03.297568 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297494    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-data" (OuterVolumeSpecName: "istio-data") pod "5ecb1e06-a25f-4093-b64b-5d5b41638b24" (UID: "5ecb1e06-a25f-4093-b64b-5d5b41638b24"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:03.297568 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297448    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5ecb1e06-a25f-4093-b64b-5d5b41638b24" (UID: "5ecb1e06-a25f-4093-b64b-5d5b41638b24"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:03.297769 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297600    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5ecb1e06-a25f-4093-b64b-5d5b41638b24" (UID: "5ecb1e06-a25f-4093-b64b-5d5b41638b24"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:03.297769 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297683    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5ecb1e06-a25f-4093-b64b-5d5b41638b24" (UID: "5ecb1e06-a25f-4093-b64b-5d5b41638b24"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:03.297881 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297820    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:03.297881 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297834    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:03.297881 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297842    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:03.297881 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297851    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:03.297881 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.297863    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:03.299392 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.299374    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5ecb1e06-a25f-4093-b64b-5d5b41638b24" (UID: "5ecb1e06-a25f-4093-b64b-5d5b41638b24"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:21:03.299490 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.299458    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5ecb1e06-a25f-4093-b64b-5d5b41638b24" (UID: "5ecb1e06-a25f-4093-b64b-5d5b41638b24"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:03.299714 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.299685    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-token" (OuterVolumeSpecName: "istio-token") pod "5ecb1e06-a25f-4093-b64b-5d5b41638b24" (UID: "5ecb1e06-a25f-4093-b64b-5d5b41638b24"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:21:03.299714 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.299709    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-kube-api-access-svstp" (OuterVolumeSpecName: "kube-api-access-svstp") pod "5ecb1e06-a25f-4093-b64b-5d5b41638b24" (UID: "5ecb1e06-a25f-4093-b64b-5d5b41638b24"). InnerVolumeSpecName "kube-api-access-svstp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:21:03.328337 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.328305    2559 generic.go:356] "Generic (PLEG): container finished" podID="5ecb1e06-a25f-4093-b64b-5d5b41638b24" containerID="060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9" exitCode=0
Sep 11 01:21:03.328481 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.328352    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"
Sep 11 01:21:03.328481 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.328390    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s" event={"ID":"5ecb1e06-a25f-4093-b64b-5d5b41638b24","Type":"ContainerDied","Data":"060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9"}
Sep 11 01:21:03.328481 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.328447    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s" event={"ID":"5ecb1e06-a25f-4093-b64b-5d5b41638b24","Type":"ContainerDied","Data":"ebea5d453e2df38eab902c14d0878f95abaad09b62c1b8a5649e04002d7ecf85"}
Sep 11 01:21:03.328481 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.328474    2559 scope.go:117] "RemoveContainer" containerID="060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9"
Sep 11 01:21:03.338543 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.338506    2559 scope.go:117] "RemoveContainer" containerID="060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9"
Sep 11 01:21:03.338825 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:21:03.338804    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9\": container with ID starting with 060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9 not found: ID does not exist" containerID="060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9"
Sep 11 01:21:03.338883 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.338834    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9"} err="failed to get container status \"060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9\": rpc error: code = NotFound desc = could not find container \"060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9\": container with ID starting with 060944461bc6beaaca0725b0ba0920bf10194deec9f9a8e29c5be123ebf860b9 not found: ID does not exist"
Sep 11 01:21:03.351257 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.351220    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"]
Sep 11 01:21:03.362189 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.357104    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-4ec6e9-openshift-default-788667c47b-9ts4s"]
Sep 11 01:21:03.398892 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.398863    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:03.398892 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.398889    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:03.399063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.398898    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svstp\" (UniqueName: \"kubernetes.io/projected/5ecb1e06-a25f-4093-b64b-5d5b41638b24-kube-api-access-svstp\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:03.399063 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:03.398909    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ecb1e06-a25f-4093-b64b-5d5b41638b24-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:05.111800 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:05.111768    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ecb1e06-a25f-4093-b64b-5d5b41638b24" path="/var/lib/kubelet/pods/5ecb1e06-a25f-4093-b64b-5d5b41638b24/volumes"
Sep 11 01:21:09.815964 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:09.815928    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"]
Sep 11 01:21:09.816436 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:09.816203    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy" containerID="cri-o://816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff" gracePeriod=30
Sep 11 01:21:15.075582 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.075556    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:21:15.126900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.126822    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-certs\") pod \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") "
Sep 11 01:21:15.126900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.126855    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-credential-socket\") pod \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") "
Sep 11 01:21:15.126900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.126893    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-envoy\") pod \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") "
Sep 11 01:21:15.127160 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.126923    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-socket\") pod \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") "
Sep 11 01:21:15.127160 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.126970    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-token\") pod \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") "
Sep 11 01:21:15.127160 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127030    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-podinfo\") pod \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") "
Sep 11 01:21:15.127160 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127095    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ncjp\" (UniqueName: \"kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-kube-api-access-9ncjp\") pod \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") "
Sep 11 01:21:15.127160 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127127    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-data\") pod \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") "
Sep 11 01:21:15.127160 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127118    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a63be8e-9c0a-49c4-9768-e5dde759b2bf" (UID: "1a63be8e-9c0a-49c4-9768-e5dde759b2bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:15.127475 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127178    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a63be8e-9c0a-49c4-9768-e5dde759b2bf" (UID: "1a63be8e-9c0a-49c4-9768-e5dde759b2bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:15.127475 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127189    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istiod-ca-cert\") pod \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\" (UID: \"1a63be8e-9c0a-49c4-9768-e5dde759b2bf\") "
Sep 11 01:21:15.127475 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127285    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a63be8e-9c0a-49c4-9768-e5dde759b2bf" (UID: "1a63be8e-9c0a-49c4-9768-e5dde759b2bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:15.127475 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127385    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a63be8e-9c0a-49c4-9768-e5dde759b2bf" (UID: "1a63be8e-9c0a-49c4-9768-e5dde759b2bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:15.127705 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127476    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a63be8e-9c0a-49c4-9768-e5dde759b2bf" (UID: "1a63be8e-9c0a-49c4-9768-e5dde759b2bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:21:15.127971 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127944    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:15.127971 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.127973    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:15.128189 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.128009    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:15.128189 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.128032    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:15.128189 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.128047    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:15.129385 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.129355    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a63be8e-9c0a-49c4-9768-e5dde759b2bf" (UID: "1a63be8e-9c0a-49c4-9768-e5dde759b2bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:15.129637 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.129581    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-kube-api-access-9ncjp" (OuterVolumeSpecName: "kube-api-access-9ncjp") pod "1a63be8e-9c0a-49c4-9768-e5dde759b2bf" (UID: "1a63be8e-9c0a-49c4-9768-e5dde759b2bf"). InnerVolumeSpecName "kube-api-access-9ncjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:21:15.129831 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.129810    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a63be8e-9c0a-49c4-9768-e5dde759b2bf" (UID: "1a63be8e-9c0a-49c4-9768-e5dde759b2bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:21:15.129917 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.129876    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a63be8e-9c0a-49c4-9768-e5dde759b2bf" (UID: "1a63be8e-9c0a-49c4-9768-e5dde759b2bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:21:15.229634 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.229575    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:15.229634 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.229632    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:15.229634 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.229642    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:15.229851 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.229651    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ncjp\" (UniqueName: \"kubernetes.io/projected/1a63be8e-9c0a-49c4-9768-e5dde759b2bf-kube-api-access-9ncjp\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:15.405642 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.405520    2559 generic.go:356] "Generic (PLEG): container finished" podID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerID="816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff" exitCode=0
Sep 11 01:21:15.405642 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.405575    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"
Sep 11 01:21:15.405843 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.405638    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" event={"ID":"1a63be8e-9c0a-49c4-9768-e5dde759b2bf","Type":"ContainerDied","Data":"816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff"}
Sep 11 01:21:15.405843 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.405693    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm" event={"ID":"1a63be8e-9c0a-49c4-9768-e5dde759b2bf","Type":"ContainerDied","Data":"e7c0bf66f5ed79479886a5b783e08b0c30d9f8c2f17ca3c6a483bd76a0fe1cf0"}
Sep 11 01:21:15.405843 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.405715    2559 scope.go:117] "RemoveContainer" containerID="816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff"
Sep 11 01:21:15.415515 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.415499    2559 scope.go:117] "RemoveContainer" containerID="db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc"
Sep 11 01:21:15.423889 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.423869    2559 scope.go:117] "RemoveContainer" containerID="816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff"
Sep 11 01:21:15.424223 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:21:15.424198    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff\": container with ID starting with 816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff not found: ID does not exist" containerID="816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff"
Sep 11 01:21:15.424382 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.424246    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff"} err="failed to get container status \"816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff\": rpc error: code = NotFound desc = could not find container \"816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff\": container with ID starting with 816d0a46153526c1e9f2f238d3ca52f7aeae934e17531df6f51886a71031ecff not found: ID does not exist"
Sep 11 01:21:15.424382 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.424263    2559 scope.go:117] "RemoveContainer" containerID="db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc"
Sep 11 01:21:15.424928 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:21:15.424736    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc\": container with ID starting with db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc not found: ID does not exist" containerID="db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc"
Sep 11 01:21:15.424928 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.424770    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc"} err="failed to get container status \"db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc\": rpc error: code = NotFound desc = could not find container \"db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc\": container with ID starting with db3e5a79d371a9474cb2d7d6d8525cd2a8f450d89ec5a405516a529c5df634dc not found: ID does not exist"
Sep 11 01:21:15.428874 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.428630    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"]
Sep 11 01:21:15.431682 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:15.431662    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e98d1e-openshift-default-995569d9d-lrzsm"]
Sep 11 01:21:17.110532 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:17.110498    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" path="/var/lib/kubelet/pods/1a63be8e-9c0a-49c4-9768-e5dde759b2bf/volumes"
Sep 11 01:21:20.630059 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.630017    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"]
Sep 11 01:21:20.630503 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.630255    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64" podUID="a3bbbf11-216e-4615-85ef-d10e967c8c23" containerName="maas-api" containerID="cri-o://6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0" gracePeriod=30
Sep 11 01:21:20.885703 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.885578    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:21:20.991036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.988714    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vrww\" (UniqueName: \"kubernetes.io/projected/a3bbbf11-216e-4615-85ef-d10e967c8c23-kube-api-access-9vrww\") pod \"a3bbbf11-216e-4615-85ef-d10e967c8c23\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") "
Sep 11 01:21:20.991036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.988778    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-metrics-certs\") pod \"a3bbbf11-216e-4615-85ef-d10e967c8c23\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") "
Sep 11 01:21:20.991036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.988845    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-maas-api-tls\") pod \"a3bbbf11-216e-4615-85ef-d10e967c8c23\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") "
Sep 11 01:21:20.991036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.988942    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbbf11-216e-4615-85ef-d10e967c8c23-openshift-service-ca\") pod \"a3bbbf11-216e-4615-85ef-d10e967c8c23\" (UID: \"a3bbbf11-216e-4615-85ef-d10e967c8c23\") "
Sep 11 01:21:20.991036 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.989895    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3bbbf11-216e-4615-85ef-d10e967c8c23-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a3bbbf11-216e-4615-85ef-d10e967c8c23" (UID: "a3bbbf11-216e-4615-85ef-d10e967c8c23"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:21:20.996415 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.996376    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3bbbf11-216e-4615-85ef-d10e967c8c23-kube-api-access-9vrww" (OuterVolumeSpecName: "kube-api-access-9vrww") pod "a3bbbf11-216e-4615-85ef-d10e967c8c23" (UID: "a3bbbf11-216e-4615-85ef-d10e967c8c23"). InnerVolumeSpecName "kube-api-access-9vrww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:21:20.998441 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.998416    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a3bbbf11-216e-4615-85ef-d10e967c8c23" (UID: "a3bbbf11-216e-4615-85ef-d10e967c8c23"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:21:20.998528 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:20.998433    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a3bbbf11-216e-4615-85ef-d10e967c8c23" (UID: "a3bbbf11-216e-4615-85ef-d10e967c8c23"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:21:21.090245 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.090202    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3bbbf11-216e-4615-85ef-d10e967c8c23-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:21.090245 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.090235    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vrww\" (UniqueName: \"kubernetes.io/projected/a3bbbf11-216e-4615-85ef-d10e967c8c23-kube-api-access-9vrww\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:21.090245 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.090250    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:21.090489 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.090261    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3bbbf11-216e-4615-85ef-d10e967c8c23-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:21.443470 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.443439    2559 generic.go:356] "Generic (PLEG): container finished" podID="a3bbbf11-216e-4615-85ef-d10e967c8c23" containerID="6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0" exitCode=0
Sep 11 01:21:21.443675 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.443499    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"
Sep 11 01:21:21.443675 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.443525    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64" event={"ID":"a3bbbf11-216e-4615-85ef-d10e967c8c23","Type":"ContainerDied","Data":"6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0"}
Sep 11 01:21:21.443675 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.443585    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64" event={"ID":"a3bbbf11-216e-4615-85ef-d10e967c8c23","Type":"ContainerDied","Data":"0628d5935737d56421b24b218f1bb274a6d699072f221de29dc1d97e847d1a67"}
Sep 11 01:21:21.443675 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.443629    2559 scope.go:117] "RemoveContainer" containerID="6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0"
Sep 11 01:21:21.453137 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.453118    2559 scope.go:117] "RemoveContainer" containerID="6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0"
Sep 11 01:21:21.453418 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:21:21.453398    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0\": container with ID starting with 6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0 not found: ID does not exist" containerID="6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0"
Sep 11 01:21:21.453469 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.453428    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0"} err="failed to get container status \"6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0\": rpc error: code = NotFound desc = could not find container \"6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0\": container with ID starting with 6e9690ff1e89266148d5241db8e6f1fab04c1637ff056ad65286307daed89df0 not found: ID does not exist"
Sep 11 01:21:21.461990 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.461966    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"]
Sep 11 01:21:21.465935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:21.465912    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-10a215-57fc4b444d-tsw64"]
Sep 11 01:21:23.110598 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:23.110554    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3bbbf11-216e-4615-85ef-d10e967c8c23" path="/var/lib/kubelet/pods/a3bbbf11-216e-4615-85ef-d10e967c8c23/volumes"
Sep 11 01:21:27.263850 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:27.263814    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"]
Sep 11 01:21:27.264337 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:27.264101    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy" containerID="cri-o://5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2" gracePeriod=30
Sep 11 01:21:28.238306 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.238275    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"]
Sep 11 01:21:28.238806 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.238787    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74872bf5-d708-4f26-8527-57077dd9813b" containerName="maas-api"
Sep 11 01:21:28.238904 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.238813    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="74872bf5-d708-4f26-8527-57077dd9813b" containerName="maas-api"
Sep 11 01:21:28.238904 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.238829    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3bbbf11-216e-4615-85ef-d10e967c8c23" containerName="maas-api"
Sep 11 01:21:28.238904 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.238838    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3bbbf11-216e-4615-85ef-d10e967c8c23" containerName="maas-api"
Sep 11 01:21:28.238904 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.238863    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy"
Sep 11 01:21:28.238904 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.238872    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy"
Sep 11 01:21:28.239124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.238911    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ecb1e06-a25f-4093-b64b-5d5b41638b24" containerName="istio-proxy"
Sep 11 01:21:28.239124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.238920    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ecb1e06-a25f-4093-b64b-5d5b41638b24" containerName="istio-proxy"
Sep 11 01:21:28.239124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.239077    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ecb1e06-a25f-4093-b64b-5d5b41638b24" containerName="istio-proxy"
Sep 11 01:21:28.239124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.239096    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy"
Sep 11 01:21:28.239124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.239107    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="74872bf5-d708-4f26-8527-57077dd9813b" containerName="maas-api"
Sep 11 01:21:28.239124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.239119    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy"
Sep 11 01:21:28.239124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.239129    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3bbbf11-216e-4615-85ef-d10e967c8c23" containerName="maas-api"
Sep 11 01:21:28.243100 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.243081    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.246097 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.246079    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-d6e26b-openshift-default-dockercfg-q5z9b\""
Sep 11 01:21:28.257993 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.257968    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"]
Sep 11 01:21:28.364506 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.364473    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-podinfo\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.364879 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.364515    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-credential-socket\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.364879 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.364543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-data\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.364879 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.364646    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istiod-ca-cert\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.364879 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.364677    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-token\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.364879 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.364782    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-envoy\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.364879 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.364823    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-socket\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.365088 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.364911    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-certs\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.365088 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.364933    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqdz7\" (UniqueName: \"kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-kube-api-access-pqdz7\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466034 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.465999    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-envoy\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466048    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-socket\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-certs\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqdz7\" (UniqueName: \"kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-kube-api-access-pqdz7\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466174    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-podinfo\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466201    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-credential-socket\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466230    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-data\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istiod-ca-cert\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466370    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-token\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466696 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466548    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-certs\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466696 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466565    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-data\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466696 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466659    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-credential-socket\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.466818 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.466745    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-socket\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.467215 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.467190    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istiod-ca-cert\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.468565 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.468538    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-envoy\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.468870 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.468851    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-podinfo\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.474413 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.474391    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-token\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.474493 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.474452    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqdz7\" (UniqueName: \"kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-kube-api-access-pqdz7\") pod \"e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") " pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.553438 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.553403    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:28.845703 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.845679    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"]
Sep 11 01:21:28.847513 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:21:28.847485    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c73e0d3_eb1d_4a07_b7ad_6a96a13dfcb5.slice/crio-797e5c1ea53cf8a8f0dc43fba45b5499f0661c192ad01db0f669851df602ca9d WatchSource:0}: Error finding container 797e5c1ea53cf8a8f0dc43fba45b5499f0661c192ad01db0f669851df602ca9d: Status 404 returned error can't find the container with id 797e5c1ea53cf8a8f0dc43fba45b5499f0661c192ad01db0f669851df602ca9d
Sep 11 01:21:28.850285 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.850249    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:21:28.850365 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.850316    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:21:28.850365 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:28.850347    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:21:29.498252 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:29.498215    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" event={"ID":"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5","Type":"ContainerStarted","Data":"ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc"}
Sep 11 01:21:29.498680 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:29.498280    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" event={"ID":"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5","Type":"ContainerStarted","Data":"797e5c1ea53cf8a8f0dc43fba45b5499f0661c192ad01db0f669851df602ca9d"}
Sep 11 01:21:29.520516 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:29.520455    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" podStartSLOduration=1.5204402510000001 podStartE2EDuration="1.520440251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:21:29.517401528 +0000 UTC m=+1409.032001710" watchObservedRunningTime="2026-09-11 01:21:29.520440251 +0000 UTC m=+1409.035040444"
Sep 11 01:21:29.554017 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:29.553974    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:29.555671 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:29.555646    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 11 01:21:29.555825 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:29.555704    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 11 01:21:30.558422 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:30.558394    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:31.510546 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:31.510486    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:31.511436 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:31.511414    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:32.518295 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.518271    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:21:32.518679 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.518657    2559 generic.go:356] "Generic (PLEG): container finished" podID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerID="5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2" exitCode=0
Sep 11 01:21:32.518743 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.518725    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" event={"ID":"be3fa414-e05b-4a3a-aedf-571a724ca39f","Type":"ContainerDied","Data":"5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2"}
Sep 11 01:21:32.518783 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.518774    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh" event={"ID":"be3fa414-e05b-4a3a-aedf-571a724ca39f","Type":"ContainerDied","Data":"854bd6e25800f575b6210a0b98ff7c286c92080e59fba5c35bdf9da6e958de79"}
Sep 11 01:21:32.518817 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.518794    2559 scope.go:117] "RemoveContainer" containerID="5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2"
Sep 11 01:21:32.531383 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.531359    2559 scope.go:117] "RemoveContainer" containerID="a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4"
Sep 11 01:21:32.541900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.541876    2559 scope.go:117] "RemoveContainer" containerID="5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2"
Sep 11 01:21:32.542245 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:21:32.542222    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2\": container with ID starting with 5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2 not found: ID does not exist" containerID="5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2"
Sep 11 01:21:32.542350 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.542258    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2"} err="failed to get container status \"5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2\": rpc error: code = NotFound desc = could not find container \"5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2\": container with ID starting with 5aedf14e7e5c3d53c91820f4889c3a9d4452b3151f0c23f63369e68d46786cd2 not found: ID does not exist"
Sep 11 01:21:32.542350 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.542283    2559 scope.go:117] "RemoveContainer" containerID="a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4"
Sep 11 01:21:32.542622 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:21:32.542582    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4\": container with ID starting with a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4 not found: ID does not exist" containerID="a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4"
Sep 11 01:21:32.542706 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.542633    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4"} err="failed to get container status \"a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4\": rpc error: code = NotFound desc = could not find container \"a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4\": container with ID starting with a893839b280ee989152411e61152df7e2998d644363c7c6318eb059b891e25b4 not found: ID does not exist"
Sep 11 01:21:32.608353 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608320    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-envoy\") pod \"be3fa414-e05b-4a3a-aedf-571a724ca39f\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") "
Sep 11 01:21:32.608556 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608371    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-credential-socket\") pod \"be3fa414-e05b-4a3a-aedf-571a724ca39f\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") "
Sep 11 01:21:32.608556 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608391    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-certs\") pod \"be3fa414-e05b-4a3a-aedf-571a724ca39f\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") "
Sep 11 01:21:32.608556 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608423    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-socket\") pod \"be3fa414-e05b-4a3a-aedf-571a724ca39f\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") "
Sep 11 01:21:32.608556 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608459    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be3fa414-e05b-4a3a-aedf-571a724ca39f-istiod-ca-cert\") pod \"be3fa414-e05b-4a3a-aedf-571a724ca39f\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") "
Sep 11 01:21:32.608556 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608521    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-data\") pod \"be3fa414-e05b-4a3a-aedf-571a724ca39f\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") "
Sep 11 01:21:32.608868 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608592    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf2vm\" (UniqueName: \"kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-kube-api-access-gf2vm\") pod \"be3fa414-e05b-4a3a-aedf-571a724ca39f\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") "
Sep 11 01:21:32.608868 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608671    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-token\") pod \"be3fa414-e05b-4a3a-aedf-571a724ca39f\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") "
Sep 11 01:21:32.608868 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608703    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "be3fa414-e05b-4a3a-aedf-571a724ca39f" (UID: "be3fa414-e05b-4a3a-aedf-571a724ca39f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:32.608868 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608723    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-podinfo\") pod \"be3fa414-e05b-4a3a-aedf-571a724ca39f\" (UID: \"be3fa414-e05b-4a3a-aedf-571a724ca39f\") "
Sep 11 01:21:32.608868 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608720    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "be3fa414-e05b-4a3a-aedf-571a724ca39f" (UID: "be3fa414-e05b-4a3a-aedf-571a724ca39f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:32.609116 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.608995    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-data" (OuterVolumeSpecName: "istio-data") pod "be3fa414-e05b-4a3a-aedf-571a724ca39f" (UID: "be3fa414-e05b-4a3a-aedf-571a724ca39f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:32.609116 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.609083    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "be3fa414-e05b-4a3a-aedf-571a724ca39f" (UID: "be3fa414-e05b-4a3a-aedf-571a724ca39f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:32.609218 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.609154    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be3fa414-e05b-4a3a-aedf-571a724ca39f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "be3fa414-e05b-4a3a-aedf-571a724ca39f" (UID: "be3fa414-e05b-4a3a-aedf-571a724ca39f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:21:32.609547 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.609512    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:32.609654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.609553    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:32.609654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.609570    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:32.609654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.609584    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be3fa414-e05b-4a3a-aedf-571a724ca39f-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:32.609654 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.609598    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:32.610881 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.610857    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "be3fa414-e05b-4a3a-aedf-571a724ca39f" (UID: "be3fa414-e05b-4a3a-aedf-571a724ca39f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:21:32.610950 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.610910    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-kube-api-access-gf2vm" (OuterVolumeSpecName: "kube-api-access-gf2vm") pod "be3fa414-e05b-4a3a-aedf-571a724ca39f" (UID: "be3fa414-e05b-4a3a-aedf-571a724ca39f"). InnerVolumeSpecName "kube-api-access-gf2vm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:21:32.611136 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.611119    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "be3fa414-e05b-4a3a-aedf-571a724ca39f" (UID: "be3fa414-e05b-4a3a-aedf-571a724ca39f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:21:32.611179 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.611124    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-token" (OuterVolumeSpecName: "istio-token") pod "be3fa414-e05b-4a3a-aedf-571a724ca39f" (UID: "be3fa414-e05b-4a3a-aedf-571a724ca39f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:21:32.710280 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.710243    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf2vm\" (UniqueName: \"kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-kube-api-access-gf2vm\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:32.710280 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.710276    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:32.710280 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.710286    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:32.710519 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.710294    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be3fa414-e05b-4a3a-aedf-571a724ca39f-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:21:32.802297 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.802211    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"]
Sep 11 01:21:32.802783 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.802764    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy"
Sep 11 01:21:32.802783 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.802782    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy"
Sep 11 01:21:32.802959 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.802799    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy"
Sep 11 01:21:32.802959 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.802805    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a63be8e-9c0a-49c4-9768-e5dde759b2bf" containerName="istio-proxy"
Sep 11 01:21:32.802959 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.802817    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy"
Sep 11 01:21:32.802959 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.802823    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy"
Sep 11 01:21:32.802959 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.802906    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy"
Sep 11 01:21:32.807331 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.807310    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:32.810323 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.810302    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-d6e26b-serving-cert\""
Sep 11 01:21:32.818860 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.818793    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"]
Sep 11 01:21:32.915169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.912919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:32.915169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.912978    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b106c2fc-1721-41fe-bdb8-73f150af08a7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:32.915169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.913020    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:32.915169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:32.913131    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mksz\" (UniqueName: \"kubernetes.io/projected/b106c2fc-1721-41fe-bdb8-73f150af08a7-kube-api-access-7mksz\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:33.014290 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.014253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:33.014290 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.014293    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b106c2fc-1721-41fe-bdb8-73f150af08a7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:33.014561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.014321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:33.014561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.014380    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mksz\" (UniqueName: \"kubernetes.io/projected/b106c2fc-1721-41fe-bdb8-73f150af08a7-kube-api-access-7mksz\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:33.014561 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:21:33.014401    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-serving-cert: secret "maas-api-e2e-shared-a-w3-d6e26b-serving-cert" not found
Sep 11 01:21:33.014561 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:21:33.014477    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls podName:b106c2fc-1721-41fe-bdb8-73f150af08a7 nodeName:}" failed. No retries permitted until 2026-09-11 01:21:33.514459081 +0000 UTC m=+1413.029059251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls") pod "maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm" (UID: "b106c2fc-1721-41fe-bdb8-73f150af08a7") : secret "maas-api-e2e-shared-a-w3-d6e26b-serving-cert" not found
Sep 11 01:21:33.015061 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.015038    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b106c2fc-1721-41fe-bdb8-73f150af08a7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:33.016795 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.016779    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:33.024865 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.024845    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mksz\" (UniqueName: \"kubernetes.io/projected/b106c2fc-1721-41fe-bdb8-73f150af08a7-kube-api-access-7mksz\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:33.523060 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.523022    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:33.523550 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:21:33.523263    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-serving-cert: secret "maas-api-e2e-shared-a-w3-d6e26b-serving-cert" not found
Sep 11 01:21:33.523550 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:21:33.523330    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls podName:b106c2fc-1721-41fe-bdb8-73f150af08a7 nodeName:}" failed. No retries permitted until 2026-09-11 01:21:34.52331077 +0000 UTC m=+1414.037910952 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls") pod "maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm" (UID: "b106c2fc-1721-41fe-bdb8-73f150af08a7") : secret "maas-api-e2e-shared-a-w3-d6e26b-serving-cert" not found
Sep 11 01:21:33.533070 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.533014    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"
Sep 11 01:21:33.561349 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.561248    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"]
Sep 11 01:21:33.563922 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:33.563834    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-10a215-openshift-default-6d646bd59b-98jhh"]
Sep 11 01:21:34.535663 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:34.535622    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:34.538228 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:34.538199    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:34.621829 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:34.621789    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:34.893103 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:34.893072    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"]
Sep 11 01:21:34.893639 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:21:34.893585    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb106c2fc_1721_41fe_bdb8_73f150af08a7.slice/crio-9b2fcdd91c2383d529d4f7a0125d5ead790023d57261b2deb8bbb058a8d8877e WatchSource:0}: Error finding container 9b2fcdd91c2383d529d4f7a0125d5ead790023d57261b2deb8bbb058a8d8877e: Status 404 returned error can't find the container with id 9b2fcdd91c2383d529d4f7a0125d5ead790023d57261b2deb8bbb058a8d8877e
Sep 11 01:21:35.115307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:35.115275    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" path="/var/lib/kubelet/pods/be3fa414-e05b-4a3a-aedf-571a724ca39f/volumes"
Sep 11 01:21:35.547698 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:35.547660    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm" event={"ID":"b106c2fc-1721-41fe-bdb8-73f150af08a7","Type":"ContainerStarted","Data":"ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c"}
Sep 11 01:21:35.548088 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:35.547723    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm" event={"ID":"b106c2fc-1721-41fe-bdb8-73f150af08a7","Type":"ContainerStarted","Data":"9b2fcdd91c2383d529d4f7a0125d5ead790023d57261b2deb8bbb058a8d8877e"}
Sep 11 01:21:35.548088 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:35.547874    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:35.568241 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:35.568187    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm" podStartSLOduration=3.135101784 podStartE2EDuration="3.568170373s" totalImagesPullingTime="433.068589ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:21:35.564709379 +0000 UTC m=+1415.079309565" watchObservedRunningTime="2026-09-11 01:21:35.568170373 +0000 UTC m=+1415.082770554"
Sep 11 01:21:41.559159 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:41.559128    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:21:53.657693 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:53.657658    2559 generic.go:356] "Generic (PLEG): container finished" podID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerID="ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc" exitCode=0
Sep 11 01:21:53.658094 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:53.657729    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" event={"ID":"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5","Type":"ContainerDied","Data":"ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc"}
Sep 11 01:21:53.658140 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:53.658119    2559 scope.go:117] "RemoveContainer" containerID="ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc"
Sep 11 01:21:53.659317 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:53.659290    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:21:53.659379 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:53.659357    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:21:53.659414 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:53.659386    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:21:54.668023 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:54.667990    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" event={"ID":"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5","Type":"ContainerStarted","Data":"6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13"}
Sep 11 01:21:55.553656 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:55.553584    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:21:55.553998 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:55.553973    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 11 01:21:55.554189 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:55.554165    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 11 01:21:56.554743 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:56.554697    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 11 01:21:56.555247 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:56.554765    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 11 01:21:57.047009 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.046956    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"]
Sep 11 01:21:57.057217 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.057186    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="be3fa414-e05b-4a3a-aedf-571a724ca39f" containerName="istio-proxy"
Sep 11 01:21:57.063627 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.063220    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"]
Sep 11 01:21:57.063627 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.063341    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.070073 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.069230    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-b9930f-openshift-default-dockercfg-wgqm5\""
Sep 11 01:21:57.192242 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.192206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-token\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.192419 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.192252    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-podinfo\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.192419 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.192289    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vzc4\" (UniqueName: \"kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-kube-api-access-4vzc4\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.192419 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.192315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294f2c10-5c4f-4c6a-a942-6d092188e168-istiod-ca-cert\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.192419 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.192346    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-data\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.192661 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.192448    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-certs\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.192661 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.192506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-credential-socket\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.192661 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.192543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-socket\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.192661 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.192562    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-envoy\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293356 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293316    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-certs\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293554 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-credential-socket\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293554 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293408    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-socket\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293554 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293429    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-envoy\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293659    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-token\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293718    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-podinfo\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-certs\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293767    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vzc4\" (UniqueName: \"kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-kube-api-access-4vzc4\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293802    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-credential-socket\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293847    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294f2c10-5c4f-4c6a-a942-6d092188e168-istiod-ca-cert\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.293941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.293900    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-data\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.294277 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.294240    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-data\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.294564 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.294542    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294f2c10-5c4f-4c6a-a942-6d092188e168-istiod-ca-cert\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.294661 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.294614    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-socket\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.295985 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.295964    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-envoy\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.296214 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.296195    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-podinfo\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.301844 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.301788    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-token\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.302120 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.302099    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vzc4\" (UniqueName: \"kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-kube-api-access-4vzc4\") pod \"e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") " pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.381127 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.381092    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:57.562805 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.562708    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 11 01:21:57.562805 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.562782    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 11 01:21:57.687399 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.687369    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"]
Sep 11 01:21:57.689538 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:21:57.689499    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294f2c10_5c4f_4c6a_a942_6d092188e168.slice/crio-5e5eec2f52dc279a8eeb6408b64aeb39036eaaf4ba7d16c8a9aedd1620248ed8 WatchSource:0}: Error finding container 5e5eec2f52dc279a8eeb6408b64aeb39036eaaf4ba7d16c8a9aedd1620248ed8: Status 404 returned error can't find the container with id 5e5eec2f52dc279a8eeb6408b64aeb39036eaaf4ba7d16c8a9aedd1620248ed8
Sep 11 01:21:57.691843 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.691809    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:21:57.691927 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.691870    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:21:57.691927 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:57.691896    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:21:58.554103 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:58.554057    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 11 01:21:58.554329 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:58.554129    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 11 01:21:58.699655 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:58.699594    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" event={"ID":"294f2c10-5c4f-4c6a-a942-6d092188e168","Type":"ContainerStarted","Data":"9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7"}
Sep 11 01:21:58.700120 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:58.699682    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" event={"ID":"294f2c10-5c4f-4c6a-a942-6d092188e168","Type":"ContainerStarted","Data":"5e5eec2f52dc279a8eeb6408b64aeb39036eaaf4ba7d16c8a9aedd1620248ed8"}
Sep 11 01:21:58.723614 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:58.723545    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" podStartSLOduration=1.723527948 podStartE2EDuration="1.723527948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:21:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:21:58.722070138 +0000 UTC m=+1438.236670340" watchObservedRunningTime="2026-09-11 01:21:58.723527948 +0000 UTC m=+1438.238128142"
Sep 11 01:21:59.381921 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:59.381885    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:59.387134 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:59.387109    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:59.554504 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:59.554462    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Sep 11 01:21:59.554815 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:59.554538    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Sep 11 01:21:59.706334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:59.706259    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:21:59.707153 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:21:59.707135    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:22:00.557935 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:00.557909    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:22:00.558221 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:00.558198    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:22:00.558990 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:00.558971    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:22:01.888639 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:01.888583    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"]
Sep 11 01:22:01.893888 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:01.893864    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:01.897477 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:01.897449    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-b9930f-serving-cert\""
Sep 11 01:22:01.909897 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:01.909873    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"]
Sep 11 01:22:01.947543 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:01.947505    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9rr7\" (UniqueName: \"kubernetes.io/projected/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-kube-api-access-m9rr7\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:01.947726 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:01.947569    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:01.947726 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:01.947648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:01.947726 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:01.947699    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:02.049008 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:02.048978    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:02.049182 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:02.049057    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:02.049182 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:02.049160    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9rr7\" (UniqueName: \"kubernetes.io/projected/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-kube-api-access-m9rr7\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:02.049257 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:22:02.049193    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-serving-cert: secret "maas-api-e2e-shared-b-w3-b9930f-serving-cert" not found
Sep 11 01:22:02.049257 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:02.049221    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:02.049334 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:22:02.049263    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-maas-api-tls podName:8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0 nodeName:}" failed. No retries permitted until 2026-09-11 01:22:02.549240788 +0000 UTC m=+1442.063840964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-maas-api-tls") pod "maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg" (UID: "8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0") : secret "maas-api-e2e-shared-b-w3-b9930f-serving-cert" not found
Sep 11 01:22:02.049858 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:02.049837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:02.051566 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:02.051539    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:02.080089 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:02.080061    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9rr7\" (UniqueName: \"kubernetes.io/projected/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-kube-api-access-m9rr7\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:02.554127 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:02.554091    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:02.556531 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:02.556509    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:02.805317 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:02.805235    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:03.082557 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:03.082522    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"]
Sep 11 01:22:03.083580 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:22:03.083552    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba4c6a2_94a6_465e_acbd_2ba7058bdfb0.slice/crio-ae7acb61634bb4d243b9757e8284dd8e45532af9626b6a7031080e70967462e2 WatchSource:0}: Error finding container ae7acb61634bb4d243b9757e8284dd8e45532af9626b6a7031080e70967462e2: Status 404 returned error can't find the container with id ae7acb61634bb4d243b9757e8284dd8e45532af9626b6a7031080e70967462e2
Sep 11 01:22:03.743529 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:03.743448    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg" event={"ID":"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0","Type":"ContainerStarted","Data":"79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c"}
Sep 11 01:22:03.743529 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:03.743497    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg" event={"ID":"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0","Type":"ContainerStarted","Data":"ae7acb61634bb4d243b9757e8284dd8e45532af9626b6a7031080e70967462e2"}
Sep 11 01:22:03.743747 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:03.743591    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:03.762558 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:03.762510    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg" podStartSLOduration=2.410395349 podStartE2EDuration="2.762496826s" totalImagesPullingTime="352.101477ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:22:03.759734731 +0000 UTC m=+1443.274334916" watchObservedRunningTime="2026-09-11 01:22:03.762496826 +0000 UTC m=+1443.277097009"
Sep 11 01:22:09.755213 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:09.755180    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:22:29.708408 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:29.708369    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 11 01:22:29.708842 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:29.708438    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 11 01:22:29.915358 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:29.915324    2559 generic.go:356] "Generic (PLEG): container finished" podID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerID="9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7" exitCode=0
Sep 11 01:22:29.915523 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:29.915377    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" event={"ID":"294f2c10-5c4f-4c6a-a942-6d092188e168","Type":"ContainerDied","Data":"9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7"}
Sep 11 01:22:29.915878 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:29.915860    2559 scope.go:117] "RemoveContainer" containerID="9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7"
Sep 11 01:22:29.917044 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:29.917013    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:22:29.917098 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:29.917086    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:22:29.917134 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:29.917117    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:22:30.125174 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.125137    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"]
Sep 11 01:22:30.126096 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.125472    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls" podUID="f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b" containerName="maas-api" containerID="cri-o://473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332" gracePeriod=30
Sep 11 01:22:30.393744 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.393714    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:22:30.472125 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.471022    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-openshift-service-ca\") pod \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") "
Sep 11 01:22:30.472125 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.471070    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-maas-api-tls\") pod \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") "
Sep 11 01:22:30.472125 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.471137    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-metrics-certs\") pod \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") "
Sep 11 01:22:30.472125 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.471292    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45grw\" (UniqueName: \"kubernetes.io/projected/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-kube-api-access-45grw\") pod \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\" (UID: \"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b\") "
Sep 11 01:22:30.481085 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.475047    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b" (UID: "f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:22:30.481558 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.481501    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b" (UID: "f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:22:30.482311 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.482233    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-kube-api-access-45grw" (OuterVolumeSpecName: "kube-api-access-45grw") pod "f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b" (UID: "f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b"). InnerVolumeSpecName "kube-api-access-45grw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:22:30.483455 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.483423    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b" (UID: "f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:22:30.573112 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.573081    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45grw\" (UniqueName: \"kubernetes.io/projected/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-kube-api-access-45grw\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:22:30.573112 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.573109    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:22:30.573112 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.573120    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:22:30.573340 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.573128    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:22:30.923435 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.923339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" event={"ID":"294f2c10-5c4f-4c6a-a942-6d092188e168","Type":"ContainerStarted","Data":"1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89"}
Sep 11 01:22:30.924710 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.924686    2559 generic.go:356] "Generic (PLEG): container finished" podID="f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b" containerID="473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332" exitCode=0
Sep 11 01:22:30.924815 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.924726    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"
Sep 11 01:22:30.924815 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.924766    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls" event={"ID":"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b","Type":"ContainerDied","Data":"473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332"}
Sep 11 01:22:30.924815 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.924812    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls" event={"ID":"f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b","Type":"ContainerDied","Data":"ae703ee818bdeb93734aa4c81402385dde31063b1fffa8cb89e6eba460a2eb9f"}
Sep 11 01:22:30.924950 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.924833    2559 scope.go:117] "RemoveContainer" containerID="473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332"
Sep 11 01:22:30.942874 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.942852    2559 scope.go:117] "RemoveContainer" containerID="473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332"
Sep 11 01:22:30.943243 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:22:30.943221    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332\": container with ID starting with 473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332 not found: ID does not exist" containerID="473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332"
Sep 11 01:22:30.943348 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.943253    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332"} err="failed to get container status \"473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332\": rpc error: code = NotFound desc = could not find container \"473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332\": container with ID starting with 473c44cc50cfbb3a46242e41612a56ddb008edb14ff8870a56f2680f83729332 not found: ID does not exist"
Sep 11 01:22:30.958177 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.958149    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"]
Sep 11 01:22:30.960643 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:30.960622    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6f69f6e6-586c97566f-z4wls"]
Sep 11 01:22:31.110860 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:31.110822    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b" path="/var/lib/kubelet/pods/f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b/volumes"
Sep 11 01:22:31.381236 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:31.381207    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:22:31.381451 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:31.381426    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 11 01:22:31.381530 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:31.381478    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 11 01:22:32.382424 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:32.382385    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 11 01:22:32.382865 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:32.382466    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 11 01:22:33.382071 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:33.382029    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 11 01:22:33.382254 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:33.382096    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 11 01:22:34.382308 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:34.382267    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 11 01:22:34.382814 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:34.382343    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 11 01:22:35.381926 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:35.381893    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 11 01:22:35.382091 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:35.381950    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 11 01:22:36.382000 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:36.381964    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 11 01:22:36.382372 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:36.382031    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 11 01:22:37.386492 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:37.386457    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:22:37.386897 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:37.386862    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:22:37.387524 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:22:37.387502    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:23:24.175012 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.174934    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"]
Sep 11 01:23:24.175430 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.175416    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b" containerName="maas-api"
Sep 11 01:23:24.175474 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.175432    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b" containerName="maas-api"
Sep 11 01:23:24.175515 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.175506    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f43c0a4c-5982-4bb9-bbf2-5c1a2f08878b" containerName="maas-api"
Sep 11 01:23:24.180087 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.180065    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.183981 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.183953    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-b9930f\"/\"auth-test-model-b9930f-kserve-self-signed-certs\""
Sep 11 01:23:24.184122 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.183985    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-b9930f\"/\"openshift-service-ca.crt\""
Sep 11 01:23:24.184281 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.184259    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-b9930f\"/\"default-dockercfg-tzbh8\""
Sep 11 01:23:24.184405 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.184353    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-b9930f\"/\"kube-root-ca.crt\""
Sep 11 01:23:24.186907 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.186884    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"]
Sep 11 01:23:24.254902 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.254866    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-home\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.254902 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.254905    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tls-certs\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.255121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.255012    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-dshm\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.255121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.255041    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdvw2\" (UniqueName: \"kubernetes.io/projected/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-kube-api-access-qdvw2\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.255121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.255107    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tmp-dir\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.255226 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.255140    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-model-cache\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.356557 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.356514    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tmp-dir\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.356769 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.356564    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-model-cache\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.356769 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.356626    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-home\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.356884 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.356766    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tls-certs\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.356946 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.356908    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-dshm\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.356994 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.356948    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tmp-dir\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.356994 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.356952    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdvw2\" (UniqueName: \"kubernetes.io/projected/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-kube-api-access-qdvw2\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.357101 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.356991    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-model-cache\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.357101 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.357035    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-home\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.359187 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.359161    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-dshm\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.359382 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.359361    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tls-certs\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.365334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.365312    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdvw2\" (UniqueName: \"kubernetes.io/projected/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-kube-api-access-qdvw2\") pod \"auth-test-model-b9930f-kserve-7df575b4f5-gsxjs\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") " pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.492589 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.492493    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:24.765341 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:24.765314    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"]
Sep 11 01:23:24.766750 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:23:24.766713    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27a2b7f0_4f8a_4098_a0ef_870967b7f8f1.slice/crio-c1ee96d20358bbb8ae98c6a4b72267148607075294aae23fa5fcbf36b9efabf6 WatchSource:0}: Error finding container c1ee96d20358bbb8ae98c6a4b72267148607075294aae23fa5fcbf36b9efabf6: Status 404 returned error can't find the container with id c1ee96d20358bbb8ae98c6a4b72267148607075294aae23fa5fcbf36b9efabf6
Sep 11 01:23:25.293495 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:25.293459    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs" event={"ID":"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1","Type":"ContainerStarted","Data":"281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715"}
Sep 11 01:23:25.293495 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:25.293501    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs" event={"ID":"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1","Type":"ContainerStarted","Data":"c1ee96d20358bbb8ae98c6a4b72267148607075294aae23fa5fcbf36b9efabf6"}
Sep 11 01:23:25.313241 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:25.313189    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs" podStartSLOduration=1.313173478 podStartE2EDuration="1.313173478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:23:25.309854189 +0000 UTC m=+1524.824454371" watchObservedRunningTime="2026-09-11 01:23:25.313173478 +0000 UTC m=+1524.827773678"
Sep 11 01:23:34.493496 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:34.493458    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:34.493990 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:34.493523    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:34.506341 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:34.506315    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:35.369557 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:35.369526    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:23:43.693687 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:43.693654    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"]
Sep 11 01:23:58.714368 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:58.714308    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg" podUID="d5b40cc7-b454-45cd-9206-d6927dd9c632" containerName="main" containerID="cri-o://09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122" gracePeriod=45
Sep 11 01:23:58.964342 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:58.964320    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:23:59.130680 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.130648    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5b40cc7-b454-45cd-9206-d6927dd9c632-tls-certs\") pod \"d5b40cc7-b454-45cd-9206-d6927dd9c632\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") "
Sep 11 01:23:59.130864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.130735    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-dshm\") pod \"d5b40cc7-b454-45cd-9206-d6927dd9c632\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") "
Sep 11 01:23:59.130864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.130761    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-tmp-dir\") pod \"d5b40cc7-b454-45cd-9206-d6927dd9c632\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") "
Sep 11 01:23:59.130864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.130792    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-model-cache\") pod \"d5b40cc7-b454-45cd-9206-d6927dd9c632\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") "
Sep 11 01:23:59.130864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.130851    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k522h\" (UniqueName: \"kubernetes.io/projected/d5b40cc7-b454-45cd-9206-d6927dd9c632-kube-api-access-k522h\") pod \"d5b40cc7-b454-45cd-9206-d6927dd9c632\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") "
Sep 11 01:23:59.131093 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.130928    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-home\") pod \"d5b40cc7-b454-45cd-9206-d6927dd9c632\" (UID: \"d5b40cc7-b454-45cd-9206-d6927dd9c632\") "
Sep 11 01:23:59.131093 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.131056    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d5b40cc7-b454-45cd-9206-d6927dd9c632" (UID: "d5b40cc7-b454-45cd-9206-d6927dd9c632"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.131184 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.131112    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-model-cache" (OuterVolumeSpecName: "model-cache") pod "d5b40cc7-b454-45cd-9206-d6927dd9c632" (UID: "d5b40cc7-b454-45cd-9206-d6927dd9c632"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.131235 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.131215    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-home" (OuterVolumeSpecName: "home") pod "d5b40cc7-b454-45cd-9206-d6927dd9c632" (UID: "d5b40cc7-b454-45cd-9206-d6927dd9c632"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.131402 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.131381    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-tmp-dir\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.131490 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.131406    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-model-cache\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.131490 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.131420    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-home\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.133026 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.133005    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-dshm" (OuterVolumeSpecName: "dshm") pod "d5b40cc7-b454-45cd-9206-d6927dd9c632" (UID: "d5b40cc7-b454-45cd-9206-d6927dd9c632"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:23:59.133139 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.133039    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5b40cc7-b454-45cd-9206-d6927dd9c632-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5b40cc7-b454-45cd-9206-d6927dd9c632" (UID: "d5b40cc7-b454-45cd-9206-d6927dd9c632"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:23:59.133139 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.133103    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5b40cc7-b454-45cd-9206-d6927dd9c632-kube-api-access-k522h" (OuterVolumeSpecName: "kube-api-access-k522h") pod "d5b40cc7-b454-45cd-9206-d6927dd9c632" (UID: "d5b40cc7-b454-45cd-9206-d6927dd9c632"). InnerVolumeSpecName "kube-api-access-k522h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:23:59.232792 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.232743    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5b40cc7-b454-45cd-9206-d6927dd9c632-tls-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.232792 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.232783    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5b40cc7-b454-45cd-9206-d6927dd9c632-dshm\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.232792 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.232796    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k522h\" (UniqueName: \"kubernetes.io/projected/d5b40cc7-b454-45cd-9206-d6927dd9c632-kube-api-access-k522h\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:23:59.509140 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.509055    2559 generic.go:356] "Generic (PLEG): container finished" podID="d5b40cc7-b454-45cd-9206-d6927dd9c632" containerID="09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122" exitCode=0
Sep 11 01:23:59.509140 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.509101    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg" event={"ID":"d5b40cc7-b454-45cd-9206-d6927dd9c632","Type":"ContainerDied","Data":"09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122"}
Sep 11 01:23:59.509140 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.509131    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg" event={"ID":"d5b40cc7-b454-45cd-9206-d6927dd9c632","Type":"ContainerDied","Data":"68311fc2b96efc76066bcd881ded90afb189296a5c668c4b5ba739c15e95ffd3"}
Sep 11 01:23:59.509374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.509148    2559 scope.go:117] "RemoveContainer" containerID="09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122"
Sep 11 01:23:59.509374 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.509104    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"
Sep 11 01:23:59.519402 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.519385    2559 scope.go:117] "RemoveContainer" containerID="09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122"
Sep 11 01:23:59.519699 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:23:59.519679    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122\": container with ID starting with 09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122 not found: ID does not exist" containerID="09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122"
Sep 11 01:23:59.519741 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.519709    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122"} err="failed to get container status \"09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122\": rpc error: code = NotFound desc = could not find container \"09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122\": container with ID starting with 09203d2c15edeb8080ad7c49826895d22beaaef896ea395e66d254b62371c122 not found: ID does not exist"
Sep 11 01:23:59.532937 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.532915    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"]
Sep 11 01:23:59.536754 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:23:59.536732    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-f4de10/unconfigured-w0-f4de10-kserve-865c4b5c5b-5xsjg"]
Sep 11 01:24:01.110768 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:01.110733    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5b40cc7-b454-45cd-9206-d6927dd9c632" path="/var/lib/kubelet/pods/d5b40cc7-b454-45cd-9206-d6927dd9c632/volumes"
Sep 11 01:24:01.714526 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:01.714482    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"]
Sep 11 01:24:01.714742 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:01.714720    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy" containerID="cri-o://f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0" gracePeriod=30
Sep 11 01:24:01.996953 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:01.996870    2559 patch_prober.go:28] interesting pod/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 01:24:01.996953 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:01.996927    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 01:24:06.964997 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:06.964954    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:24:07.017777 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.017741    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-socket\") pod \"69ade794-505f-4668-9d61-3703316ccb1c\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") "
Sep 11 01:24:07.017964 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.017871    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-data\") pod \"69ade794-505f-4668-9d61-3703316ccb1c\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") "
Sep 11 01:24:07.017964 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.017913    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kg9p\" (UniqueName: \"kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-kube-api-access-7kg9p\") pod \"69ade794-505f-4668-9d61-3703316ccb1c\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") "
Sep 11 01:24:07.017964 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.017951    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69ade794-505f-4668-9d61-3703316ccb1c-istiod-ca-cert\") pod \"69ade794-505f-4668-9d61-3703316ccb1c\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") "
Sep 11 01:24:07.018138 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.017975    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69ade794-505f-4668-9d61-3703316ccb1c" (UID: "69ade794-505f-4668-9d61-3703316ccb1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:24:07.018138 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018038    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69ade794-505f-4668-9d61-3703316ccb1c-istio-podinfo\") pod \"69ade794-505f-4668-9d61-3703316ccb1c\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") "
Sep 11 01:24:07.018138 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018092    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-credential-socket\") pod \"69ade794-505f-4668-9d61-3703316ccb1c\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") "
Sep 11 01:24:07.018138 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018121    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-certs\") pod \"69ade794-505f-4668-9d61-3703316ccb1c\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") "
Sep 11 01:24:07.018342 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018157    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-istio-token\") pod \"69ade794-505f-4668-9d61-3703316ccb1c\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") "
Sep 11 01:24:07.018342 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018116    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-data" (OuterVolumeSpecName: "istio-data") pod "69ade794-505f-4668-9d61-3703316ccb1c" (UID: "69ade794-505f-4668-9d61-3703316ccb1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:24:07.018342 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018202    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-envoy\") pod \"69ade794-505f-4668-9d61-3703316ccb1c\" (UID: \"69ade794-505f-4668-9d61-3703316ccb1c\") "
Sep 11 01:24:07.018501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018387    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69ade794-505f-4668-9d61-3703316ccb1c" (UID: "69ade794-505f-4668-9d61-3703316ccb1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:24:07.018501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018434    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69ade794-505f-4668-9d61-3703316ccb1c" (UID: "69ade794-505f-4668-9d61-3703316ccb1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:24:07.018501 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018445    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ade794-505f-4668-9d61-3703316ccb1c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69ade794-505f-4668-9d61-3703316ccb1c" (UID: "69ade794-505f-4668-9d61-3703316ccb1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:24:07.018719 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018703    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:24:07.018781 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018721    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:24:07.018781 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018730    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69ade794-505f-4668-9d61-3703316ccb1c-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:24:07.018781 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018740    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:24:07.018781 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.018752    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:24:07.020200 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.020175    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69ade794-505f-4668-9d61-3703316ccb1c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69ade794-505f-4668-9d61-3703316ccb1c" (UID: "69ade794-505f-4668-9d61-3703316ccb1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:24:07.020311 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.020270    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-istio-token" (OuterVolumeSpecName: "istio-token") pod "69ade794-505f-4668-9d61-3703316ccb1c" (UID: "69ade794-505f-4668-9d61-3703316ccb1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:24:07.020569 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.020547    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-kube-api-access-7kg9p" (OuterVolumeSpecName: "kube-api-access-7kg9p") pod "69ade794-505f-4668-9d61-3703316ccb1c" (UID: "69ade794-505f-4668-9d61-3703316ccb1c"). InnerVolumeSpecName "kube-api-access-7kg9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:24:07.020569 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.020554    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69ade794-505f-4668-9d61-3703316ccb1c" (UID: "69ade794-505f-4668-9d61-3703316ccb1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:24:07.119530 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.119507    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69ade794-505f-4668-9d61-3703316ccb1c-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:24:07.119530 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.119531    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:24:07.119717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.119543    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69ade794-505f-4668-9d61-3703316ccb1c-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:24:07.119717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.119555    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kg9p\" (UniqueName: \"kubernetes.io/projected/69ade794-505f-4668-9d61-3703316ccb1c-kube-api-access-7kg9p\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:24:07.563580 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.563549    2559 generic.go:356] "Generic (PLEG): container finished" podID="69ade794-505f-4668-9d61-3703316ccb1c" containerID="f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0" exitCode=0
Sep 11 01:24:07.563788 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.563596    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"
Sep 11 01:24:07.563788 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.563636    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" event={"ID":"69ade794-505f-4668-9d61-3703316ccb1c","Type":"ContainerDied","Data":"f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0"}
Sep 11 01:24:07.563788 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.563698    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9" event={"ID":"69ade794-505f-4668-9d61-3703316ccb1c","Type":"ContainerDied","Data":"b40cc579c1e8c2eb529bb40327d3f4385b53cb603adb89fba071dc15ab69bdef"}
Sep 11 01:24:07.563788 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.563730    2559 scope.go:117] "RemoveContainer" containerID="f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0"
Sep 11 01:24:07.573494 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.573476    2559 scope.go:117] "RemoveContainer" containerID="c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944"
Sep 11 01:24:07.582367 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.582334    2559 scope.go:117] "RemoveContainer" containerID="f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0"
Sep 11 01:24:07.582639 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:24:07.582616    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0\": container with ID starting with f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0 not found: ID does not exist" containerID="f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0"
Sep 11 01:24:07.582717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.582648    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0"} err="failed to get container status \"f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0\": rpc error: code = NotFound desc = could not find container \"f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0\": container with ID starting with f567197d3b86eb2da397eb9874f07fe827b733d221c8d452fda424bb37cc8dc0 not found: ID does not exist"
Sep 11 01:24:07.582717 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.582664    2559 scope.go:117] "RemoveContainer" containerID="c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944"
Sep 11 01:24:07.582918 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.582898    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"]
Sep 11 01:24:07.582985 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:24:07.582904    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944\": container with ID starting with c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944 not found: ID does not exist" containerID="c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944"
Sep 11 01:24:07.582985 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.582960    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944"} err="failed to get container status \"c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944\": rpc error: code = NotFound desc = could not find container \"c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944\": container with ID starting with c5127f2bb2001bc2f419cb79100550b975e604992cf60245e125331bf2dfa944 not found: ID does not exist"
Sep 11 01:24:07.589551 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:07.589527    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f4de10-openshift-default-7d6bb4864-hqmb9"]
Sep 11 01:24:09.111149 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:09.111115    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ade794-505f-4668-9d61-3703316ccb1c" path="/var/lib/kubelet/pods/69ade794-505f-4668-9d61-3703316ccb1c/volumes"
Sep 11 01:24:26.306423 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.306391    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"]
Sep 11 01:24:26.306872 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.306859    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy"
Sep 11 01:24:26.306934 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.306874    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy"
Sep 11 01:24:26.306934 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.306888    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy"
Sep 11 01:24:26.306934 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.306896    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy"
Sep 11 01:24:26.306934 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.306904    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5b40cc7-b454-45cd-9206-d6927dd9c632" containerName="main"
Sep 11 01:24:26.306934 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.306910    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5b40cc7-b454-45cd-9206-d6927dd9c632" containerName="main"
Sep 11 01:24:26.307103 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.306978    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5b40cc7-b454-45cd-9206-d6927dd9c632" containerName="main"
Sep 11 01:24:26.307103 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.306987    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy"
Sep 11 01:24:26.307103 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.306994    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ade794-505f-4668-9d61-3703316ccb1c" containerName="istio-proxy"
Sep 11 01:24:26.315370 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.315337    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.318930 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.318522    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f729f8-serving-cert\""
Sep 11 01:24:26.323558 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.323532    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"]
Sep 11 01:24:26.418887 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.418850    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d23dd51-5670-4916-b149-35c6a7d2a7f5-openshift-service-ca\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.419064 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.418914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-metrics-certs\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.419064 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.419015    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jlkt\" (UniqueName: \"kubernetes.io/projected/5d23dd51-5670-4916-b149-35c6a7d2a7f5-kube-api-access-4jlkt\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.419064 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.419047    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-maas-api-tls\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.520746 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.520706    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jlkt\" (UniqueName: \"kubernetes.io/projected/5d23dd51-5670-4916-b149-35c6a7d2a7f5-kube-api-access-4jlkt\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.520746 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.520749    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-maas-api-tls\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.521020 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.520839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d23dd51-5670-4916-b149-35c6a7d2a7f5-openshift-service-ca\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.521020 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.520889    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-metrics-certs\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.521536 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.521499    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d23dd51-5670-4916-b149-35c6a7d2a7f5-openshift-service-ca\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.523294 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.523273    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-metrics-certs\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.523452 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.523428    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-maas-api-tls\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.529728 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.529702    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jlkt\" (UniqueName: \"kubernetes.io/projected/5d23dd51-5670-4916-b149-35c6a7d2a7f5-kube-api-access-4jlkt\") pod \"maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.633160 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.633013    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:26.914720 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.914685    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"]
Sep 11 01:24:26.915910 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:24:26.915878    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d23dd51_5670_4916_b149_35c6a7d2a7f5.slice/crio-a4221d292b6d3f6c1db871ca2f714c0a36bb5ed050379b1335288c5a3a1ff155 WatchSource:0}: Error finding container a4221d292b6d3f6c1db871ca2f714c0a36bb5ed050379b1335288c5a3a1ff155: Status 404 returned error can't find the container with id a4221d292b6d3f6c1db871ca2f714c0a36bb5ed050379b1335288c5a3a1ff155
Sep 11 01:24:26.917376 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:26.917357    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 01:24:27.688781 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:27.688681    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x" event={"ID":"5d23dd51-5670-4916-b149-35c6a7d2a7f5","Type":"ContainerStarted","Data":"1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54"}
Sep 11 01:24:27.688781 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:27.688737    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x" event={"ID":"5d23dd51-5670-4916-b149-35c6a7d2a7f5","Type":"ContainerStarted","Data":"a4221d292b6d3f6c1db871ca2f714c0a36bb5ed050379b1335288c5a3a1ff155"}
Sep 11 01:24:27.689261 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:27.688778    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:24:27.706590 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:27.706532    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x" podStartSLOduration=1.264009847 podStartE2EDuration="1.706518022s" totalImagesPullingTime="442.508175ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:24:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:24:27.703837344 +0000 UTC m=+1587.218437527" watchObservedRunningTime="2026-09-11 01:24:27.706518022 +0000 UTC m=+1587.221118204"
Sep 11 01:24:33.700046 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:24:33.700014    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:25:38.452589 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.452555    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"]
Sep 11 01:25:38.453123 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.452817    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x" podUID="5d23dd51-5670-4916-b149-35c6a7d2a7f5" containerName="maas-api" containerID="cri-o://1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54" gracePeriod=30
Sep 11 01:25:38.711121 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.711052    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:25:38.801886 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.801839    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jlkt\" (UniqueName: \"kubernetes.io/projected/5d23dd51-5670-4916-b149-35c6a7d2a7f5-kube-api-access-4jlkt\") pod \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") "
Sep 11 01:25:38.802080 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.801944    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d23dd51-5670-4916-b149-35c6a7d2a7f5-openshift-service-ca\") pod \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") "
Sep 11 01:25:38.802080 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.802018    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-metrics-certs\") pod \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") "
Sep 11 01:25:38.802198 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.802101    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-maas-api-tls\") pod \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\" (UID: \"5d23dd51-5670-4916-b149-35c6a7d2a7f5\") "
Sep 11 01:25:38.802474 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.802436    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d23dd51-5670-4916-b149-35c6a7d2a7f5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d23dd51-5670-4916-b149-35c6a7d2a7f5" (UID: "5d23dd51-5670-4916-b149-35c6a7d2a7f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:25:38.804217 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.804178    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5d23dd51-5670-4916-b149-35c6a7d2a7f5" (UID: "5d23dd51-5670-4916-b149-35c6a7d2a7f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:25:38.804217 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.804208    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d23dd51-5670-4916-b149-35c6a7d2a7f5" (UID: "5d23dd51-5670-4916-b149-35c6a7d2a7f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:25:38.804348 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.804239    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d23dd51-5670-4916-b149-35c6a7d2a7f5-kube-api-access-4jlkt" (OuterVolumeSpecName: "kube-api-access-4jlkt") pod "5d23dd51-5670-4916-b149-35c6a7d2a7f5" (UID: "5d23dd51-5670-4916-b149-35c6a7d2a7f5"). InnerVolumeSpecName "kube-api-access-4jlkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:25:38.903960 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.903925    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jlkt\" (UniqueName: \"kubernetes.io/projected/5d23dd51-5670-4916-b149-35c6a7d2a7f5-kube-api-access-4jlkt\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:25:38.903960 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.903954    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d23dd51-5670-4916-b149-35c6a7d2a7f5-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:25:38.903960 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.903964    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:25:38.904210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:38.903973    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d23dd51-5670-4916-b149-35c6a7d2a7f5-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:25:39.125510 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:39.125481    2559 generic.go:356] "Generic (PLEG): container finished" podID="5d23dd51-5670-4916-b149-35c6a7d2a7f5" containerID="1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54" exitCode=0
Sep 11 01:25:39.125674 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:39.125526    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"
Sep 11 01:25:39.125674 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:39.125562    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x" event={"ID":"5d23dd51-5670-4916-b149-35c6a7d2a7f5","Type":"ContainerDied","Data":"1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54"}
Sep 11 01:25:39.125674 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:39.125636    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x" event={"ID":"5d23dd51-5670-4916-b149-35c6a7d2a7f5","Type":"ContainerDied","Data":"a4221d292b6d3f6c1db871ca2f714c0a36bb5ed050379b1335288c5a3a1ff155"}
Sep 11 01:25:39.125674 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:39.125666    2559 scope.go:117] "RemoveContainer" containerID="1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54"
Sep 11 01:25:39.135384 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:39.135355    2559 scope.go:117] "RemoveContainer" containerID="1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54"
Sep 11 01:25:39.135709 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:25:39.135685    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54\": container with ID starting with 1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54 not found: ID does not exist" containerID="1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54"
Sep 11 01:25:39.135822 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:39.135714    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54"} err="failed to get container status \"1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54\": rpc error: code = NotFound desc = could not find container \"1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54\": container with ID starting with 1a8f9f1339f559b8f2bdb22f633b691de61e2af91f9061f875784f930736fc54 not found: ID does not exist"
Sep 11 01:25:39.143592 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:39.143570    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"]
Sep 11 01:25:39.147525 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:39.147503    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x"]
Sep 11 01:25:41.112713 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:41.112681    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d23dd51-5670-4916-b149-35c6a7d2a7f5" path="/var/lib/kubelet/pods/5d23dd51-5670-4916-b149-35c6a7d2a7f5/volumes"
Sep 11 01:25:41.694995 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:41.694947    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f729f8-7b9b9dfd54-4pc5x" podUID="5d23dd51-5670-4916-b149-35c6a7d2a7f5" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.78:8443/health\": context deadline exceeded"
Sep 11 01:25:58.560048 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:25:58.560014    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"]
Sep 11 01:26:13.579822 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.579759    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs" podUID="27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" containerName="main" containerID="cri-o://281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715" gracePeriod=45
Sep 11 01:26:13.834965 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.834909    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:26:13.974979 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.974927    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tls-certs\") pod \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") "
Sep 11 01:26:13.975171 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975020    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdvw2\" (UniqueName: \"kubernetes.io/projected/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-kube-api-access-qdvw2\") pod \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") "
Sep 11 01:26:13.975171 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975109    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-dshm\") pod \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") "
Sep 11 01:26:13.975171 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975138    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tmp-dir\") pod \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") "
Sep 11 01:26:13.975318 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975233    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-model-cache\") pod \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") "
Sep 11 01:26:13.975377 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975357    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-home\") pod \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\" (UID: \"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1\") "
Sep 11 01:26:13.975475 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975448    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" (UID: "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:13.975547 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975504    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" (UID: "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:13.975682 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975659    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-home" (OuterVolumeSpecName: "home") pod "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" (UID: "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:13.975838 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975818    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-model-cache\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:13.975886 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975839    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-home\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:13.975886 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.975852    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tmp-dir\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:13.977356 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.977339    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-dshm" (OuterVolumeSpecName: "dshm") pod "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" (UID: "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:13.977536 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.977520    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" (UID: "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:26:13.977581 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:13.977564    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-kube-api-access-qdvw2" (OuterVolumeSpecName: "kube-api-access-qdvw2") pod "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" (UID: "27a2b7f0-4f8a-4098-a0ef-870967b7f8f1"). InnerVolumeSpecName "kube-api-access-qdvw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:26:14.076992 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.076947    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-dshm\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:14.076992 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.076987    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-tls-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:14.076992 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.076998    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdvw2\" (UniqueName: \"kubernetes.io/projected/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1-kube-api-access-qdvw2\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:14.336389 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.336356    2559 generic.go:356] "Generic (PLEG): container finished" podID="27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" containerID="281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715" exitCode=0
Sep 11 01:26:14.336558 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.336398    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"
Sep 11 01:26:14.336558 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.336443    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs" event={"ID":"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1","Type":"ContainerDied","Data":"281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715"}
Sep 11 01:26:14.336558 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.336481    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs" event={"ID":"27a2b7f0-4f8a-4098-a0ef-870967b7f8f1","Type":"ContainerDied","Data":"c1ee96d20358bbb8ae98c6a4b72267148607075294aae23fa5fcbf36b9efabf6"}
Sep 11 01:26:14.336558 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.336505    2559 scope.go:117] "RemoveContainer" containerID="281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715"
Sep 11 01:26:14.347118 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.347104    2559 scope.go:117] "RemoveContainer" containerID="281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715"
Sep 11 01:26:14.347386 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:26:14.347349    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715\": container with ID starting with 281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715 not found: ID does not exist" containerID="281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715"
Sep 11 01:26:14.347456 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.347385    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715"} err="failed to get container status \"281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715\": rpc error: code = NotFound desc = could not find container \"281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715\": container with ID starting with 281cafdd2a5b2fcc67ac095e90111d9e8a79e300f27a0ac74fb9a0b35cce3715 not found: ID does not exist"
Sep 11 01:26:14.358871 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.358846    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"]
Sep 11 01:26:14.361772 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:14.361753    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b9930f/auth-test-model-b9930f-kserve-7df575b4f5-gsxjs"]
Sep 11 01:26:15.110790 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:15.110761    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" path="/var/lib/kubelet/pods/27a2b7f0-4f8a-4098-a0ef-870967b7f8f1/volumes"
Sep 11 01:26:18.616582 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.616548    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"]
Sep 11 01:26:18.617052 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.616816    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92" podUID="5d63e28a-7e91-4aa5-98b2-f77e49053c2d" containerName="maas-api" containerID="cri-o://b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55" gracePeriod=30
Sep 11 01:26:18.867343 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.867288    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:26:18.922960 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.922923    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-metrics-certs\") pod \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") "
Sep 11 01:26:18.922960 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.922961    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjlqd\" (UniqueName: \"kubernetes.io/projected/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-kube-api-access-mjlqd\") pod \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") "
Sep 11 01:26:18.923178 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.923008    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-openshift-service-ca\") pod \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") "
Sep 11 01:26:18.923178 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.923065    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-maas-api-tls\") pod \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\" (UID: \"5d63e28a-7e91-4aa5-98b2-f77e49053c2d\") "
Sep 11 01:26:18.923369 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.923341    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d63e28a-7e91-4aa5-98b2-f77e49053c2d" (UID: "5d63e28a-7e91-4aa5-98b2-f77e49053c2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:26:18.923478 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.923461    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:18.925064 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.925041    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d63e28a-7e91-4aa5-98b2-f77e49053c2d" (UID: "5d63e28a-7e91-4aa5-98b2-f77e49053c2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:26:18.925153 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.925098    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5d63e28a-7e91-4aa5-98b2-f77e49053c2d" (UID: "5d63e28a-7e91-4aa5-98b2-f77e49053c2d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:26:18.925153 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:18.925111    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-kube-api-access-mjlqd" (OuterVolumeSpecName: "kube-api-access-mjlqd") pod "5d63e28a-7e91-4aa5-98b2-f77e49053c2d" (UID: "5d63e28a-7e91-4aa5-98b2-f77e49053c2d"). InnerVolumeSpecName "kube-api-access-mjlqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:26:19.026117 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.025986    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:19.026117 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.026027    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjlqd\" (UniqueName: \"kubernetes.io/projected/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-kube-api-access-mjlqd\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:19.026117 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.026043    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d63e28a-7e91-4aa5-98b2-f77e49053c2d-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:19.369540 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.369455    2559 generic.go:356] "Generic (PLEG): container finished" podID="5d63e28a-7e91-4aa5-98b2-f77e49053c2d" containerID="b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55" exitCode=0
Sep 11 01:26:19.369540 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.369502    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"
Sep 11 01:26:19.369782 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.369538    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92" event={"ID":"5d63e28a-7e91-4aa5-98b2-f77e49053c2d","Type":"ContainerDied","Data":"b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55"}
Sep 11 01:26:19.369782 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.369589    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92" event={"ID":"5d63e28a-7e91-4aa5-98b2-f77e49053c2d","Type":"ContainerDied","Data":"dc19fb313829c7e285e424e59ed8144157697a772402c2fddec749954287fb14"}
Sep 11 01:26:19.369782 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.369629    2559 scope.go:117] "RemoveContainer" containerID="b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55"
Sep 11 01:26:19.378630 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.378596    2559 scope.go:117] "RemoveContainer" containerID="b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55"
Sep 11 01:26:19.378896 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:26:19.378878    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55\": container with ID starting with b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55 not found: ID does not exist" containerID="b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55"
Sep 11 01:26:19.378967 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.378904    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55"} err="failed to get container status \"b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55\": rpc error: code = NotFound desc = could not find container \"b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55\": container with ID starting with b853f2060beada6541b2e1b4f319c0fff59bb3ac7856a9e71b4b5258b87bfd55 not found: ID does not exist"
Sep 11 01:26:19.389305 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.389283    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"]
Sep 11 01:26:19.393340 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:19.393320    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-76bb56-77f44bbd5c-ksc92"]
Sep 11 01:26:21.111199 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:21.111160    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d63e28a-7e91-4aa5-98b2-f77e49053c2d" path="/var/lib/kubelet/pods/5d63e28a-7e91-4aa5-98b2-f77e49053c2d/volumes"
Sep 11 01:26:25.693890 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:25.693813    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"]
Sep 11 01:26:25.694238 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:25.694033    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy" containerID="cri-o://19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419" gracePeriod=30
Sep 11 01:26:30.957238 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:30.957180    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:26:31.043319 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043290    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-data\") pod \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") "
Sep 11 01:26:31.043504 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043336    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-token\") pod \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") "
Sep 11 01:26:31.043504 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043391    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgjz5\" (UniqueName: \"kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-kube-api-access-mgjz5\") pod \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") "
Sep 11 01:26:31.043504 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043419    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-socket\") pod \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") "
Sep 11 01:26:31.043504 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043433    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-certs\") pod \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") "
Sep 11 01:26:31.043504 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043454    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-podinfo\") pod \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") "
Sep 11 01:26:31.043504 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043471    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-envoy\") pod \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") "
Sep 11 01:26:31.043866 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043509    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-credential-socket\") pod \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") "
Sep 11 01:26:31.043866 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043594    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc0383f8-fbe2-40c6-aeb4-bae3da376212" (UID: "fc0383f8-fbe2-40c6-aeb4-bae3da376212"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:31.043866 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043640    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istiod-ca-cert\") pod \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\" (UID: \"fc0383f8-fbe2-40c6-aeb4-bae3da376212\") "
Sep 11 01:26:31.043866 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043677    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc0383f8-fbe2-40c6-aeb4-bae3da376212" (UID: "fc0383f8-fbe2-40c6-aeb4-bae3da376212"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:31.043866 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043805    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc0383f8-fbe2-40c6-aeb4-bae3da376212" (UID: "fc0383f8-fbe2-40c6-aeb4-bae3da376212"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:31.044126 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.043884    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc0383f8-fbe2-40c6-aeb4-bae3da376212" (UID: "fc0383f8-fbe2-40c6-aeb4-bae3da376212"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:31.044126 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.044024    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:31.044126 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.044041    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:31.044126 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.044052    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:31.044126 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.044064    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:31.044381 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.044128    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc0383f8-fbe2-40c6-aeb4-bae3da376212" (UID: "fc0383f8-fbe2-40c6-aeb4-bae3da376212"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:26:31.045738 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.045713    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc0383f8-fbe2-40c6-aeb4-bae3da376212" (UID: "fc0383f8-fbe2-40c6-aeb4-bae3da376212"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:26:31.045865 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.045756    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-kube-api-access-mgjz5" (OuterVolumeSpecName: "kube-api-access-mgjz5") pod "fc0383f8-fbe2-40c6-aeb4-bae3da376212" (UID: "fc0383f8-fbe2-40c6-aeb4-bae3da376212"). InnerVolumeSpecName "kube-api-access-mgjz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:26:31.046127 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.046104    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc0383f8-fbe2-40c6-aeb4-bae3da376212" (UID: "fc0383f8-fbe2-40c6-aeb4-bae3da376212"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:26:31.046178 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.046109    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc0383f8-fbe2-40c6-aeb4-bae3da376212" (UID: "fc0383f8-fbe2-40c6-aeb4-bae3da376212"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:26:31.145579 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.145550    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgjz5\" (UniqueName: \"kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-kube-api-access-mgjz5\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:31.145579 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.145576    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:31.145579 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.145585    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:31.145579 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.145593    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:31.145877 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.145619    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc0383f8-fbe2-40c6-aeb4-bae3da376212-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:26:31.441328 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.441294    2559 generic.go:356] "Generic (PLEG): container finished" podID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerID="19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419" exitCode=0
Sep 11 01:26:31.441488 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.441366    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"
Sep 11 01:26:31.441488 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.441379    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" event={"ID":"fc0383f8-fbe2-40c6-aeb4-bae3da376212","Type":"ContainerDied","Data":"19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419"}
Sep 11 01:26:31.441488 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.441458    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67" event={"ID":"fc0383f8-fbe2-40c6-aeb4-bae3da376212","Type":"ContainerDied","Data":"e0f3c7886a235f21ffe6e9e919ef21cd165022a5539da1711e02b2ccdf1201f9"}
Sep 11 01:26:31.441488 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.441481    2559 scope.go:117] "RemoveContainer" containerID="19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419"
Sep 11 01:26:31.450861 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.450840    2559 scope.go:117] "RemoveContainer" containerID="d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2"
Sep 11 01:26:31.460686 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.460588    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"]
Sep 11 01:26:31.460954 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.460933    2559 scope.go:117] "RemoveContainer" containerID="19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419"
Sep 11 01:26:31.461354 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:26:31.461329    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419\": container with ID starting with 19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419 not found: ID does not exist" containerID="19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419"
Sep 11 01:26:31.461448 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.461415    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419"} err="failed to get container status \"19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419\": rpc error: code = NotFound desc = could not find container \"19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419\": container with ID starting with 19ed29eb90b6a9db8c687cd6bec73cd2898ed55d8f2a86a622561984b6ebf419 not found: ID does not exist"
Sep 11 01:26:31.461519 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.461452    2559 scope.go:117] "RemoveContainer" containerID="d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2"
Sep 11 01:26:31.461832 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:26:31.461801    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2\": container with ID starting with d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2 not found: ID does not exist" containerID="d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2"
Sep 11 01:26:31.461908 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.461839    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2"} err="failed to get container status \"d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2\": rpc error: code = NotFound desc = could not find container \"d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2\": container with ID starting with d0b1f0740fe587abf16d80d2814d482460bef38dcc4a1aa5d7aa94e717f6c5d2 not found: ID does not exist"
Sep 11 01:26:31.464012 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:31.463995    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-76bb56-openshift-default-54985b7cfd-2pm67"]
Sep 11 01:26:33.110219 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:26:33.110182    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" path="/var/lib/kubelet/pods/fc0383f8-fbe2-40c6-aeb4-bae3da376212/volumes"
Sep 11 01:29:45.894536 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.894505    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"]
Sep 11 01:29:45.895334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895124    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy"
Sep 11 01:29:45.895334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895149    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy"
Sep 11 01:29:45.895334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895164    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy"
Sep 11 01:29:45.895334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895173    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy"
Sep 11 01:29:45.895334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895190    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d63e28a-7e91-4aa5-98b2-f77e49053c2d" containerName="maas-api"
Sep 11 01:29:45.895334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895199    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d63e28a-7e91-4aa5-98b2-f77e49053c2d" containerName="maas-api"
Sep 11 01:29:45.895334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895224    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d23dd51-5670-4916-b149-35c6a7d2a7f5" containerName="maas-api"
Sep 11 01:29:45.895334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895232    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d23dd51-5670-4916-b149-35c6a7d2a7f5" containerName="maas-api"
Sep 11 01:29:45.895334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895264    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" containerName="main"
Sep 11 01:29:45.895334 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895272    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" containerName="main"
Sep 11 01:29:45.895829 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895402    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d63e28a-7e91-4aa5-98b2-f77e49053c2d" containerName="maas-api"
Sep 11 01:29:45.895829 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895419    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d23dd51-5670-4916-b149-35c6a7d2a7f5" containerName="maas-api"
Sep 11 01:29:45.895829 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895433    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy"
Sep 11 01:29:45.895829 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895444    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc0383f8-fbe2-40c6-aeb4-bae3da376212" containerName="istio-proxy"
Sep 11 01:29:45.895829 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.895454    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="27a2b7f0-4f8a-4098-a0ef-870967b7f8f1" containerName="main"
Sep 11 01:29:45.898672 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.898655    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:45.901561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.901529    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-40f90a-serving-cert\""
Sep 11 01:29:45.910776 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.910749    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"]
Sep 11 01:29:45.985354 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.985323    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4klp2\" (UniqueName: \"kubernetes.io/projected/c7f9e923-15c2-478f-a1da-a089c118cd4e-kube-api-access-4klp2\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:45.985561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.985395    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-metrics-certs\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:45.985561 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.985498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:45.985713 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:45.985692    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7f9e923-15c2-478f-a1da-a089c118cd4e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:46.086614 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:46.086577    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7f9e923-15c2-478f-a1da-a089c118cd4e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:46.086856 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:46.086670    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4klp2\" (UniqueName: \"kubernetes.io/projected/c7f9e923-15c2-478f-a1da-a089c118cd4e-kube-api-access-4klp2\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:46.086856 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:46.086725    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-metrics-certs\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:46.086856 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:46.086760    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:46.087044 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:29:46.086963    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-serving-cert: secret "maas-api-e2e-ipp-b-40f90a-serving-cert" not found
Sep 11 01:29:46.087101 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:29:46.087048    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-maas-api-tls podName:c7f9e923-15c2-478f-a1da-a089c118cd4e nodeName:}" failed. No retries permitted until 2026-09-11 01:29:46.58702716 +0000 UTC m=+1906.101627323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-maas-api-tls") pod "maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c" (UID: "c7f9e923-15c2-478f-a1da-a089c118cd4e") : secret "maas-api-e2e-ipp-b-40f90a-serving-cert" not found
Sep 11 01:29:46.087469 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:46.087438    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7f9e923-15c2-478f-a1da-a089c118cd4e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:46.089440 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:46.089418    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-metrics-certs\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:46.100247 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:46.100226    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4klp2\" (UniqueName: \"kubernetes.io/projected/c7f9e923-15c2-478f-a1da-a089c118cd4e-kube-api-access-4klp2\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:46.591677 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:46.591642    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:46.594069 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:46.594049    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:46.811804 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:46.811771    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:47.090083 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:47.090052    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"]
Sep 11 01:29:47.090943 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:29:47.090919    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f9e923_15c2_478f_a1da_a089c118cd4e.slice/crio-cb49944dd8f772d75646d9160066d9b89a95363097c140d638921d4016ec7c1f WatchSource:0}: Error finding container cb49944dd8f772d75646d9160066d9b89a95363097c140d638921d4016ec7c1f: Status 404 returned error can't find the container with id cb49944dd8f772d75646d9160066d9b89a95363097c140d638921d4016ec7c1f
Sep 11 01:29:47.092274 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:47.092258    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 01:29:47.598223 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:47.598184    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c" event={"ID":"c7f9e923-15c2-478f-a1da-a089c118cd4e","Type":"ContainerStarted","Data":"0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af"}
Sep 11 01:29:47.598415 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:47.598237    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c" event={"ID":"c7f9e923-15c2-478f-a1da-a089c118cd4e","Type":"ContainerStarted","Data":"cb49944dd8f772d75646d9160066d9b89a95363097c140d638921d4016ec7c1f"}
Sep 11 01:29:47.598415 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:47.598323    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:29:47.625590 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:47.621350    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c" podStartSLOduration=2.298163364 podStartE2EDuration="2.621328346s" totalImagesPullingTime="323.164982ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:29:47.61792642 +0000 UTC m=+1907.132526604" watchObservedRunningTime="2026-09-11 01:29:47.621328346 +0000 UTC m=+1907.135928530"
Sep 11 01:29:53.609868 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:29:53.609839    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:30:15.121033 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:15.121000    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t27rf_8312527e-f390-4329-bd4c-e76f9cf8c530/payload-processing/0.log"
Sep 11 01:30:48.300166 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:48.300131    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t27rf_8312527e-f390-4329-bd4c-e76f9cf8c530/payload-processing/0.log"
Sep 11 01:30:48.993251 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:48.993210    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"]
Sep 11 01:30:48.997087 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:48.997064    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:48.999823 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:48.999801    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-8fa57c-openshift-default-dockercfg-wb8gg\""
Sep 11 01:30:49.008776 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.008746    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"]
Sep 11 01:30:49.128786 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.128748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30003f72-6e7b-4bbc-97fe-9e4686383515-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.128980 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.128802    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-data\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.128980 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.128823    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-token\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.128980 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.128858    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-certs\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.128980 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.128889    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-socket\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.128980 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.128951    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-envoy\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.129173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.129034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-credential-socket\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.129173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.129061    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-podinfo\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.129173 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.129086    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr67d\" (UniqueName: \"kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-kube-api-access-zr67d\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.230567 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.230526    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-credential-socket\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.230567 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.230567    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-podinfo\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.230838 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.230593    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr67d\" (UniqueName: \"kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-kube-api-access-zr67d\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.230838 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.230676    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30003f72-6e7b-4bbc-97fe-9e4686383515-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.230838 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.230755    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-data\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.230838 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.230778    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-token\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.231048 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.230988    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-certs\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.231048 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.231026    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-credential-socket\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.231048 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.231044    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-socket\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.231214 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.231108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-data\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.231214 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.231177    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-envoy\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.231325 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.231300    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-certs\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.231403 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.231380    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-socket\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.231511 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.231488    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30003f72-6e7b-4bbc-97fe-9e4686383515-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.233144 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.233120    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-podinfo\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.233301 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.233286    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-envoy\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.239153 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.239120    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-token\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.239283 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.239261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr67d\" (UniqueName: \"kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-kube-api-access-zr67d\") pod \"e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") " pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.310623 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.310574    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:49.587960 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.587927    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"]
Sep 11 01:30:49.589471 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:30:49.589443    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30003f72_6e7b_4bbc_97fe_9e4686383515.slice/crio-c9211b97948ecc8a1aa8c5119d5ba2fc82266af997b1de641e0c02f2e307125e WatchSource:0}: Error finding container c9211b97948ecc8a1aa8c5119d5ba2fc82266af997b1de641e0c02f2e307125e: Status 404 returned error can't find the container with id c9211b97948ecc8a1aa8c5119d5ba2fc82266af997b1de641e0c02f2e307125e
Sep 11 01:30:49.591899 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.591865    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:30:49.591995 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.591934    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:30:49.591995 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.591961    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:30:49.973104 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.973008    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" event={"ID":"30003f72-6e7b-4bbc-97fe-9e4686383515","Type":"ContainerStarted","Data":"534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c"}
Sep 11 01:30:49.973104 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.973102    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" event={"ID":"30003f72-6e7b-4bbc-97fe-9e4686383515","Type":"ContainerStarted","Data":"c9211b97948ecc8a1aa8c5119d5ba2fc82266af997b1de641e0c02f2e307125e"}
Sep 11 01:30:49.996495 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:49.996445    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" podStartSLOduration=1.9964301180000001 podStartE2EDuration="1.996430118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:30:49.994845726 +0000 UTC m=+1969.509445909" watchObservedRunningTime="2026-09-11 01:30:49.996430118 +0000 UTC m=+1969.511030311"
Sep 11 01:30:50.310708 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:50.310667    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:50.316570 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:50.316545    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:50.978991 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:50.978961    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:50.979879 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:50.979859    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:30:53.207405 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.207325    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"]
Sep 11 01:30:53.211382 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.211352    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.213979 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.213953    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-8fa57c-serving-cert\""
Sep 11 01:30:53.221399 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.221251    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"]
Sep 11 01:30:53.276183 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.276145    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlbqq\" (UniqueName: \"kubernetes.io/projected/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-kube-api-access-mlbqq\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.276341 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.276226    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.276341 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.276270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.276506 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.276436    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.377343 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.377306    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlbqq\" (UniqueName: \"kubernetes.io/projected/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-kube-api-access-mlbqq\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.377517 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.377360    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.377517 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.377386    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.377517 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.377467    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.377517 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:30:53.377504    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-serving-cert: secret "maas-api-e2e-ipp-cleanup-8fa57c-serving-cert" not found
Sep 11 01:30:53.377733 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:30:53.377575    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-maas-api-tls podName:4c61d5e5-6bf2-46bd-b738-5d1d6328ac20 nodeName:}" failed. No retries permitted until 2026-09-11 01:30:53.877554425 +0000 UTC m=+1973.392154599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm" (UID: "4c61d5e5-6bf2-46bd-b738-5d1d6328ac20") : secret "maas-api-e2e-ipp-cleanup-8fa57c-serving-cert" not found
Sep 11 01:30:53.378171 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.378146    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.379826 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.379809    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.385718 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.385697    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlbqq\" (UniqueName: \"kubernetes.io/projected/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-kube-api-access-mlbqq\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.883002 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.882953    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:53.885442 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:53.885417    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:54.125365 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:54.125328    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:54.408251 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:54.408222    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"]
Sep 11 01:30:54.409749 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:30:54.409720    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c61d5e5_6bf2_46bd_b738_5d1d6328ac20.slice/crio-b757f9eddc366f6efe84d1cd059b7fdb183bcdd1f1114fddf366509950dfe88c WatchSource:0}: Error finding container b757f9eddc366f6efe84d1cd059b7fdb183bcdd1f1114fddf366509950dfe88c: Status 404 returned error can't find the container with id b757f9eddc366f6efe84d1cd059b7fdb183bcdd1f1114fddf366509950dfe88c
Sep 11 01:30:55.003017 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:55.002979    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm" event={"ID":"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20","Type":"ContainerStarted","Data":"5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3"}
Sep 11 01:30:55.003197 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:55.003034    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:30:55.003197 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:55.003053    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm" event={"ID":"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20","Type":"ContainerStarted","Data":"b757f9eddc366f6efe84d1cd059b7fdb183bcdd1f1114fddf366509950dfe88c"}
Sep 11 01:30:55.024258 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:30:55.024189    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm" podStartSLOduration=1.541877501 podStartE2EDuration="2.024164656s" totalImagesPullingTime="482.287155ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:30:55.018975169 +0000 UTC m=+1974.533575363" watchObservedRunningTime="2026-09-11 01:30:55.024164656 +0000 UTC m=+1974.538764841"
Sep 11 01:31:01.013952 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:01.013920    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:31:12.116358 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:12.116323    2559 generic.go:356] "Generic (PLEG): container finished" podID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerID="534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c" exitCode=0
Sep 11 01:31:12.116794 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:12.116402    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" event={"ID":"30003f72-6e7b-4bbc-97fe-9e4686383515","Type":"ContainerDied","Data":"534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c"}
Sep 11 01:31:12.116838 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:12.116820    2559 scope.go:117] "RemoveContainer" containerID="534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c"
Sep 11 01:31:12.118163 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:12.118136    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:31:12.118228 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:12.118204    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:31:12.118265 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:12.118237    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 01:31:13.123940 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:13.123906    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" event={"ID":"30003f72-6e7b-4bbc-97fe-9e4686383515","Type":"ContainerStarted","Data":"d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e"}
Sep 11 01:31:13.311172 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:13.311130    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:31:13.311475 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:13.311448    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 11 01:31:13.311533 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:13.311513    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 11 01:31:14.311122 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:14.311085    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 11 01:31:14.311513 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:14.311138    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 11 01:31:15.311084 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:15.311046    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 11 01:31:15.311274 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:15.311136    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 11 01:31:16.311813 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:16.311774    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 11 01:31:16.312195 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:16.311839    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 11 01:31:17.311287 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:17.311039    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 11 01:31:17.311542 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:17.311298    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 11 01:31:18.350093 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:18.350061    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:31:18.350528 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:18.350363    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:31:18.351043 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:18.351022    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:31:23.795398 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:23.795362    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"]
Sep 11 01:31:23.795924 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:23.795576    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm" podUID="4c61d5e5-6bf2-46bd-b738-5d1d6328ac20" containerName="maas-api" containerID="cri-o://5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3" gracePeriod=30
Sep 11 01:31:24.072454 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.072428    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:31:24.208625 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.203894    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-openshift-service-ca\") pod \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") "
Sep 11 01:31:24.208625 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.204065    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-maas-api-tls\") pod \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") "
Sep 11 01:31:24.208625 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.204142    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlbqq\" (UniqueName: \"kubernetes.io/projected/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-kube-api-access-mlbqq\") pod \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") "
Sep 11 01:31:24.208625 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.204191    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-metrics-certs\") pod \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\" (UID: \"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20\") "
Sep 11 01:31:24.208625 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.205140    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c61d5e5-6bf2-46bd-b738-5d1d6328ac20" (UID: "4c61d5e5-6bf2-46bd-b738-5d1d6328ac20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:31:24.208625 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.205839    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:24.210124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.209534    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-kube-api-access-mlbqq" (OuterVolumeSpecName: "kube-api-access-mlbqq") pod "4c61d5e5-6bf2-46bd-b738-5d1d6328ac20" (UID: "4c61d5e5-6bf2-46bd-b738-5d1d6328ac20"). InnerVolumeSpecName "kube-api-access-mlbqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:31:24.217097 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.217040    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c61d5e5-6bf2-46bd-b738-5d1d6328ac20" (UID: "4c61d5e5-6bf2-46bd-b738-5d1d6328ac20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:31:24.218120 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.217887    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4c61d5e5-6bf2-46bd-b738-5d1d6328ac20" (UID: "4c61d5e5-6bf2-46bd-b738-5d1d6328ac20"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:31:24.222701 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.222670    2559 generic.go:356] "Generic (PLEG): container finished" podID="4c61d5e5-6bf2-46bd-b738-5d1d6328ac20" containerID="5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3" exitCode=0
Sep 11 01:31:24.222901 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.222734    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"
Sep 11 01:31:24.222901 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.222766    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm" event={"ID":"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20","Type":"ContainerDied","Data":"5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3"}
Sep 11 01:31:24.222901 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.222836    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm" event={"ID":"4c61d5e5-6bf2-46bd-b738-5d1d6328ac20","Type":"ContainerDied","Data":"b757f9eddc366f6efe84d1cd059b7fdb183bcdd1f1114fddf366509950dfe88c"}
Sep 11 01:31:24.222901 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.222861    2559 scope.go:117] "RemoveContainer" containerID="5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3"
Sep 11 01:31:24.237233 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.237212    2559 scope.go:117] "RemoveContainer" containerID="5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3"
Sep 11 01:31:24.237512 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:31:24.237494    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3\": container with ID starting with 5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3 not found: ID does not exist" containerID="5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3"
Sep 11 01:31:24.237574 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.237525    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3"} err="failed to get container status \"5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3\": rpc error: code = NotFound desc = could not find container \"5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3\": container with ID starting with 5f99a65a850acf744ad685d57542f478b0c7fd1c62f949722beccc295566dfa3 not found: ID does not exist"
Sep 11 01:31:24.248577 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.248552    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"]
Sep 11 01:31:24.251127 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.251109    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8fa57c-64684c698d-w5fbm"]
Sep 11 01:31:24.307156 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.307073    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:24.307156 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.307101    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlbqq\" (UniqueName: \"kubernetes.io/projected/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-kube-api-access-mlbqq\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:24.307156 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:24.307111    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:25.110247 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:25.110212    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c61d5e5-6bf2-46bd-b738-5d1d6328ac20" path="/var/lib/kubelet/pods/4c61d5e5-6bf2-46bd-b738-5d1d6328ac20/volumes"
Sep 11 01:31:30.662442 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:30.662410    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"]
Sep 11 01:31:30.662892 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:30.662627    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy" containerID="cri-o://d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e" gracePeriod=30
Sep 11 01:31:33.352683 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:33.352645    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 01:31:33.353073 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:33.352713    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 01:31:35.926272 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:35.926211    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:31:36.023969 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.023933    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-envoy\") pod \"30003f72-6e7b-4bbc-97fe-9e4686383515\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") "
Sep 11 01:31:36.024157 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024033    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30003f72-6e7b-4bbc-97fe-9e4686383515-istiod-ca-cert\") pod \"30003f72-6e7b-4bbc-97fe-9e4686383515\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") "
Sep 11 01:31:36.024157 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024112    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr67d\" (UniqueName: \"kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-kube-api-access-zr67d\") pod \"30003f72-6e7b-4bbc-97fe-9e4686383515\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") "
Sep 11 01:31:36.024157 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024143    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-certs\") pod \"30003f72-6e7b-4bbc-97fe-9e4686383515\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") "
Sep 11 01:31:36.024332 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024169    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-socket\") pod \"30003f72-6e7b-4bbc-97fe-9e4686383515\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") "
Sep 11 01:31:36.024332 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024196    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-credential-socket\") pod \"30003f72-6e7b-4bbc-97fe-9e4686383515\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") "
Sep 11 01:31:36.024332 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024237    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-token\") pod \"30003f72-6e7b-4bbc-97fe-9e4686383515\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") "
Sep 11 01:31:36.024332 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024262    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-podinfo\") pod \"30003f72-6e7b-4bbc-97fe-9e4686383515\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") "
Sep 11 01:31:36.024332 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024301    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-data\") pod \"30003f72-6e7b-4bbc-97fe-9e4686383515\" (UID: \"30003f72-6e7b-4bbc-97fe-9e4686383515\") "
Sep 11 01:31:36.024584 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024419    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30003f72-6e7b-4bbc-97fe-9e4686383515" (UID: "30003f72-6e7b-4bbc-97fe-9e4686383515"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:31:36.024584 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024436    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30003f72-6e7b-4bbc-97fe-9e4686383515" (UID: "30003f72-6e7b-4bbc-97fe-9e4686383515"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:31:36.024584 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024465    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30003f72-6e7b-4bbc-97fe-9e4686383515-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30003f72-6e7b-4bbc-97fe-9e4686383515" (UID: "30003f72-6e7b-4bbc-97fe-9e4686383515"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:31:36.024780 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024680    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-data" (OuterVolumeSpecName: "istio-data") pod "30003f72-6e7b-4bbc-97fe-9e4686383515" (UID: "30003f72-6e7b-4bbc-97fe-9e4686383515"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:31:36.024780 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024758    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:36.024780 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024777    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:36.024936 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024793    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30003f72-6e7b-4bbc-97fe-9e4686383515-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:36.024936 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.024832    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30003f72-6e7b-4bbc-97fe-9e4686383515" (UID: "30003f72-6e7b-4bbc-97fe-9e4686383515"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:31:36.026235 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.026210    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30003f72-6e7b-4bbc-97fe-9e4686383515" (UID: "30003f72-6e7b-4bbc-97fe-9e4686383515"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:31:36.026373 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.026347    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-token" (OuterVolumeSpecName: "istio-token") pod "30003f72-6e7b-4bbc-97fe-9e4686383515" (UID: "30003f72-6e7b-4bbc-97fe-9e4686383515"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:31:36.026447 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.026427    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30003f72-6e7b-4bbc-97fe-9e4686383515" (UID: "30003f72-6e7b-4bbc-97fe-9e4686383515"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:31:36.026523 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.026505    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-kube-api-access-zr67d" (OuterVolumeSpecName: "kube-api-access-zr67d") pod "30003f72-6e7b-4bbc-97fe-9e4686383515" (UID: "30003f72-6e7b-4bbc-97fe-9e4686383515"). InnerVolumeSpecName "kube-api-access-zr67d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:31:36.125287 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.125255    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr67d\" (UniqueName: \"kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-kube-api-access-zr67d\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:36.125287 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.125280    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:36.125287 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.125290    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:36.125560 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.125298    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:36.125560 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.125306    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:36.125560 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.125313    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30003f72-6e7b-4bbc-97fe-9e4686383515-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:31:36.302769 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.302735    2559 generic.go:356] "Generic (PLEG): container finished" podID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerID="d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e" exitCode=0
Sep 11 01:31:36.302957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.302789    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"
Sep 11 01:31:36.302957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.302831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" event={"ID":"30003f72-6e7b-4bbc-97fe-9e4686383515","Type":"ContainerDied","Data":"d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e"}
Sep 11 01:31:36.302957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.302886    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7" event={"ID":"30003f72-6e7b-4bbc-97fe-9e4686383515","Type":"ContainerDied","Data":"c9211b97948ecc8a1aa8c5119d5ba2fc82266af997b1de641e0c02f2e307125e"}
Sep 11 01:31:36.302957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.302908    2559 scope.go:117] "RemoveContainer" containerID="d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e"
Sep 11 01:31:36.312832 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.312813    2559 scope.go:117] "RemoveContainer" containerID="534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c"
Sep 11 01:31:36.321303 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.321227    2559 scope.go:117] "RemoveContainer" containerID="d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e"
Sep 11 01:31:36.321587 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:31:36.321558    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e\": container with ID starting with d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e not found: ID does not exist" containerID="d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e"
Sep 11 01:31:36.321712 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.321629    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e"} err="failed to get container status \"d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e\": rpc error: code = NotFound desc = could not find container \"d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e\": container with ID starting with d9083bde1e0d9d38e62a2412a7d0d3ef16cdc0cde9e2dd90d392ba497fb74c2e not found: ID does not exist"
Sep 11 01:31:36.321712 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.321656    2559 scope.go:117] "RemoveContainer" containerID="534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c"
Sep 11 01:31:36.322110 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:31:36.322086    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c\": container with ID starting with 534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c not found: ID does not exist" containerID="534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c"
Sep 11 01:31:36.322201 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.322114    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c"} err="failed to get container status \"534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c\": rpc error: code = NotFound desc = could not find container \"534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c\": container with ID starting with 534eeebe1fcd150d78a7cd47612dc08701f8e1131358943d72dfc8ef053c808c not found: ID does not exist"
Sep 11 01:31:36.324724 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.324706    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"]
Sep 11 01:31:36.329005 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:36.328976    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8fa57c-openshift-default-c8f44bf68-zq7q7"]
Sep 11 01:31:37.111432 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:31:37.111394    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" path="/var/lib/kubelet/pods/30003f72-6e7b-4bbc-97fe-9e4686383515/volumes"
Sep 11 01:32:49.867106 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:49.867070    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"]
Sep 11 01:32:49.867546 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:49.867304    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c" podUID="c7f9e923-15c2-478f-a1da-a089c118cd4e" containerName="maas-api" containerID="cri-o://0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af" gracePeriod=30
Sep 11 01:32:50.119286 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.119224    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:32:50.169858 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.169824    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7f9e923-15c2-478f-a1da-a089c118cd4e-openshift-service-ca\") pod \"c7f9e923-15c2-478f-a1da-a089c118cd4e\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") "
Sep 11 01:32:50.170030 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.169962    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-metrics-certs\") pod \"c7f9e923-15c2-478f-a1da-a089c118cd4e\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") "
Sep 11 01:32:50.170030 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.170007    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4klp2\" (UniqueName: \"kubernetes.io/projected/c7f9e923-15c2-478f-a1da-a089c118cd4e-kube-api-access-4klp2\") pod \"c7f9e923-15c2-478f-a1da-a089c118cd4e\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") "
Sep 11 01:32:50.170106 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.170030    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-maas-api-tls\") pod \"c7f9e923-15c2-478f-a1da-a089c118cd4e\" (UID: \"c7f9e923-15c2-478f-a1da-a089c118cd4e\") "
Sep 11 01:32:50.170226 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.170199    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7f9e923-15c2-478f-a1da-a089c118cd4e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c7f9e923-15c2-478f-a1da-a089c118cd4e" (UID: "c7f9e923-15c2-478f-a1da-a089c118cd4e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:32:50.170430 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.170399    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7f9e923-15c2-478f-a1da-a089c118cd4e-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:32:50.172032 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.172006    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f9e923-15c2-478f-a1da-a089c118cd4e-kube-api-access-4klp2" (OuterVolumeSpecName: "kube-api-access-4klp2") pod "c7f9e923-15c2-478f-a1da-a089c118cd4e" (UID: "c7f9e923-15c2-478f-a1da-a089c118cd4e"). InnerVolumeSpecName "kube-api-access-4klp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:32:50.172449 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.172428    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c7f9e923-15c2-478f-a1da-a089c118cd4e" (UID: "c7f9e923-15c2-478f-a1da-a089c118cd4e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:32:50.172511 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.172452    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c7f9e923-15c2-478f-a1da-a089c118cd4e" (UID: "c7f9e923-15c2-478f-a1da-a089c118cd4e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:32:50.271523 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.271448    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:32:50.271523 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.271485    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4klp2\" (UniqueName: \"kubernetes.io/projected/c7f9e923-15c2-478f-a1da-a089c118cd4e-kube-api-access-4klp2\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:32:50.271523 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.271497    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f9e923-15c2-478f-a1da-a089c118cd4e-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:32:50.752880 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.752849    2559 generic.go:356] "Generic (PLEG): container finished" podID="c7f9e923-15c2-478f-a1da-a089c118cd4e" containerID="0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af" exitCode=0
Sep 11 01:32:50.753057 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.752898    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"
Sep 11 01:32:50.753057 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.752938    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c" event={"ID":"c7f9e923-15c2-478f-a1da-a089c118cd4e","Type":"ContainerDied","Data":"0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af"}
Sep 11 01:32:50.753057 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.753002    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c" event={"ID":"c7f9e923-15c2-478f-a1da-a089c118cd4e","Type":"ContainerDied","Data":"cb49944dd8f772d75646d9160066d9b89a95363097c140d638921d4016ec7c1f"}
Sep 11 01:32:50.753057 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.753033    2559 scope.go:117] "RemoveContainer" containerID="0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af"
Sep 11 01:32:50.766399 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.766377    2559 scope.go:117] "RemoveContainer" containerID="0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af"
Sep 11 01:32:50.766765 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:32:50.766745    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af\": container with ID starting with 0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af not found: ID does not exist" containerID="0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af"
Sep 11 01:32:50.766856 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.766778    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af"} err="failed to get container status \"0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af\": rpc error: code = NotFound desc = could not find container \"0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af\": container with ID starting with 0b1ac66f82165fa94f263d2da10cfcda8364fa17afb2f35ad04ee17c926dd7af not found: ID does not exist"
Sep 11 01:32:50.779869 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.779827    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"]
Sep 11 01:32:50.782415 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:50.782390    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-40f90a-5b9b95f898-wvt6c"]
Sep 11 01:32:51.117122 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:32:51.117081    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7f9e923-15c2-478f-a1da-a089c118cd4e" path="/var/lib/kubelet/pods/c7f9e923-15c2-478f-a1da-a089c118cd4e/volumes"
Sep 11 01:33:07.398649 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.398597    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"]
Sep 11 01:33:07.399061 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.398882    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm" podUID="b106c2fc-1721-41fe-bdb8-73f150af08a7" containerName="maas-api" containerID="cri-o://ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c" gracePeriod=30
Sep 11 01:33:07.642629 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.642583    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:33:07.750396 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.750302    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-metrics-certs\") pod \"b106c2fc-1721-41fe-bdb8-73f150af08a7\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") "
Sep 11 01:33:07.750396 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.750363    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls\") pod \"b106c2fc-1721-41fe-bdb8-73f150af08a7\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") "
Sep 11 01:33:07.750656 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.750436    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mksz\" (UniqueName: \"kubernetes.io/projected/b106c2fc-1721-41fe-bdb8-73f150af08a7-kube-api-access-7mksz\") pod \"b106c2fc-1721-41fe-bdb8-73f150af08a7\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") "
Sep 11 01:33:07.750656 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.750478    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b106c2fc-1721-41fe-bdb8-73f150af08a7-openshift-service-ca\") pod \"b106c2fc-1721-41fe-bdb8-73f150af08a7\" (UID: \"b106c2fc-1721-41fe-bdb8-73f150af08a7\") "
Sep 11 01:33:07.751165 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.751129    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b106c2fc-1721-41fe-bdb8-73f150af08a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b106c2fc-1721-41fe-bdb8-73f150af08a7" (UID: "b106c2fc-1721-41fe-bdb8-73f150af08a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:33:07.752848 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.752820    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b106c2fc-1721-41fe-bdb8-73f150af08a7-kube-api-access-7mksz" (OuterVolumeSpecName: "kube-api-access-7mksz") pod "b106c2fc-1721-41fe-bdb8-73f150af08a7" (UID: "b106c2fc-1721-41fe-bdb8-73f150af08a7"). InnerVolumeSpecName "kube-api-access-7mksz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:33:07.753090 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.753061    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b106c2fc-1721-41fe-bdb8-73f150af08a7" (UID: "b106c2fc-1721-41fe-bdb8-73f150af08a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:33:07.753090 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.753078    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b106c2fc-1721-41fe-bdb8-73f150af08a7" (UID: "b106c2fc-1721-41fe-bdb8-73f150af08a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:33:07.851682 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.851648    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:07.851682 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.851674    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mksz\" (UniqueName: \"kubernetes.io/projected/b106c2fc-1721-41fe-bdb8-73f150af08a7-kube-api-access-7mksz\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:07.851682 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.851683    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b106c2fc-1721-41fe-bdb8-73f150af08a7-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:07.851682 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.851692    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b106c2fc-1721-41fe-bdb8-73f150af08a7-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:07.854152 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.854120    2559 generic.go:356] "Generic (PLEG): container finished" podID="b106c2fc-1721-41fe-bdb8-73f150af08a7" containerID="ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c" exitCode=0
Sep 11 01:33:07.854291 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.854165    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"
Sep 11 01:33:07.854291 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.854185    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm" event={"ID":"b106c2fc-1721-41fe-bdb8-73f150af08a7","Type":"ContainerDied","Data":"ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c"}
Sep 11 01:33:07.854291 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.854238    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm" event={"ID":"b106c2fc-1721-41fe-bdb8-73f150af08a7","Type":"ContainerDied","Data":"9b2fcdd91c2383d529d4f7a0125d5ead790023d57261b2deb8bbb058a8d8877e"}
Sep 11 01:33:07.854291 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.854268    2559 scope.go:117] "RemoveContainer" containerID="ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c"
Sep 11 01:33:07.864322 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.864305    2559 scope.go:117] "RemoveContainer" containerID="ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c"
Sep 11 01:33:07.864620 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:33:07.864586    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c\": container with ID starting with ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c not found: ID does not exist" containerID="ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c"
Sep 11 01:33:07.864699 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.864629    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c"} err="failed to get container status \"ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c\": rpc error: code = NotFound desc = could not find container \"ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c\": container with ID starting with ee24929de852503a90d3bba88e0dff4d60dbbd35ca7f6e72f0906b7337d3982c not found: ID does not exist"
Sep 11 01:33:07.877267 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.877246    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"]
Sep 11 01:33:07.882859 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:07.882838    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-d6e26b-dc758d687-zdlpm"]
Sep 11 01:33:09.109804 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:09.109763    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b106c2fc-1721-41fe-bdb8-73f150af08a7" path="/var/lib/kubelet/pods/b106c2fc-1721-41fe-bdb8-73f150af08a7/volumes"
Sep 11 01:33:14.290025 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:14.289989    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"]
Sep 11 01:33:14.290540 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:14.290198    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy" containerID="cri-o://6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13" gracePeriod=30
Sep 11 01:33:15.560716 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:15.560676    2559 patch_prober.go:28] interesting pod/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 01:33:15.561172 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:15.560746    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 01:33:19.541995 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.541969    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:33:19.568758 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.568686    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-envoy\") pod \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") "
Sep 11 01:33:19.568758 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.568731    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-certs\") pod \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") "
Sep 11 01:33:19.568961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.568759    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-token\") pod \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") "
Sep 11 01:33:19.568961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.568808    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-socket\") pod \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") "
Sep 11 01:33:19.568961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.568874    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istiod-ca-cert\") pod \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") "
Sep 11 01:33:19.569138 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.569009    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" (UID: "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:33:19.569202 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.569149    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" (UID: "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:33:19.569267 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.569248    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-podinfo\") pod \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") "
Sep 11 01:33:19.569313 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.569283    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-data\") pod \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") "
Sep 11 01:33:19.569367 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.569350    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" (UID: "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:33:19.569406 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.569350    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqdz7\" (UniqueName: \"kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-kube-api-access-pqdz7\") pod \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") "
Sep 11 01:33:19.569842 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.569817    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-data" (OuterVolumeSpecName: "istio-data") pod "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" (UID: "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:33:19.569972 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.569858    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:19.569972 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.569912    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:19.569972 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.569928    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:19.571074 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.571053    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" (UID: "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:33:19.571317 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.571283    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" (UID: "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:33:19.571621 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.571580    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" (UID: "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:33:19.571682 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.571594    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-kube-api-access-pqdz7" (OuterVolumeSpecName: "kube-api-access-pqdz7") pod "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" (UID: "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5"). InnerVolumeSpecName "kube-api-access-pqdz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:33:19.671216 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.671179    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-credential-socket\") pod \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\" (UID: \"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5\") "
Sep 11 01:33:19.671440 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.671417    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" (UID: "7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:33:19.671582 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.671570    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqdz7\" (UniqueName: \"kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-kube-api-access-pqdz7\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:19.671644 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.671584    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:19.671644 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.671594    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:19.671644 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.671627    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:19.671644 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.671641    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:19.671774 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.671652    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:19.928892 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.928800    2559 generic.go:356] "Generic (PLEG): container finished" podID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerID="6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13" exitCode=0
Sep 11 01:33:19.928892 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.928859    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"
Sep 11 01:33:19.929124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.928886    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" event={"ID":"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5","Type":"ContainerDied","Data":"6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13"}
Sep 11 01:33:19.929124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.928939    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp" event={"ID":"7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5","Type":"ContainerDied","Data":"797e5c1ea53cf8a8f0dc43fba45b5499f0661c192ad01db0f669851df602ca9d"}
Sep 11 01:33:19.929124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.928963    2559 scope.go:117] "RemoveContainer" containerID="6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13"
Sep 11 01:33:19.938632 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.938613    2559 scope.go:117] "RemoveContainer" containerID="ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc"
Sep 11 01:33:19.946704 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.946686    2559 scope.go:117] "RemoveContainer" containerID="6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13"
Sep 11 01:33:19.946961 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:33:19.946946    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13\": container with ID starting with 6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13 not found: ID does not exist" containerID="6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13"
Sep 11 01:33:19.947004 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.946970    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13"} err="failed to get container status \"6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13\": rpc error: code = NotFound desc = could not find container \"6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13\": container with ID starting with 6024fd053d0fde2ce96590853045d8456a3bf24b18d23642fcf1fef067bb8e13 not found: ID does not exist"
Sep 11 01:33:19.947004 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.946985    2559 scope.go:117] "RemoveContainer" containerID="ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc"
Sep 11 01:33:19.947179 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:33:19.947164    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc\": container with ID starting with ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc not found: ID does not exist" containerID="ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc"
Sep 11 01:33:19.947221 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.947182    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc"} err="failed to get container status \"ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc\": rpc error: code = NotFound desc = could not find container \"ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc\": container with ID starting with ca523a935f5550e8b34a7878ab1319b9bb67cafef3a30bb415291782d409bafc not found: ID does not exist"
Sep 11 01:33:19.951344 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.951321    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"]
Sep 11 01:33:19.955124 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:19.955103    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-d6e26b-openshift-default-6c8cd59c7d-blspp"]
Sep 11 01:33:21.115438 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:21.115407    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" path="/var/lib/kubelet/pods/7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5/volumes"
Sep 11 01:33:25.126376 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.126329    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"]
Sep 11 01:33:25.126864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.126587    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg" podUID="8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0" containerName="maas-api" containerID="cri-o://79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c" gracePeriod=30
Sep 11 01:33:25.370751 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.370728    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:33:25.425013 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.424933    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9rr7\" (UniqueName: \"kubernetes.io/projected/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-kube-api-access-m9rr7\") pod \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") "
Sep 11 01:33:25.425013 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.424977    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-metrics-certs\") pod \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") "
Sep 11 01:33:25.425194 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.425072    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-maas-api-tls\") pod \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") "
Sep 11 01:33:25.425194 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.425097    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-openshift-service-ca\") pod \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\" (UID: \"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0\") "
Sep 11 01:33:25.425524 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.425499    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0" (UID: "8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:33:25.427056 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.427034    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0" (UID: "8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:33:25.427377 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.427361    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0" (UID: "8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:33:25.427450 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.427391    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-kube-api-access-m9rr7" (OuterVolumeSpecName: "kube-api-access-m9rr7") pod "8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0" (UID: "8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0"). InnerVolumeSpecName "kube-api-access-m9rr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:33:25.526029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.525987    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:25.526029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.526026    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:25.526287 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.526041    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9rr7\" (UniqueName: \"kubernetes.io/projected/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-kube-api-access-m9rr7\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:25.526287 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.526057    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:25.965550 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.965512    2559 generic.go:356] "Generic (PLEG): container finished" podID="8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0" containerID="79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c" exitCode=0
Sep 11 01:33:25.965747 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.965561    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"
Sep 11 01:33:25.965747 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.965590    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg" event={"ID":"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0","Type":"ContainerDied","Data":"79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c"}
Sep 11 01:33:25.965747 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.965667    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg" event={"ID":"8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0","Type":"ContainerDied","Data":"ae7acb61634bb4d243b9757e8284dd8e45532af9626b6a7031080e70967462e2"}
Sep 11 01:33:25.965747 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.965692    2559 scope.go:117] "RemoveContainer" containerID="79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c"
Sep 11 01:33:25.975740 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.975723    2559 scope.go:117] "RemoveContainer" containerID="79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c"
Sep 11 01:33:25.976018 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:33:25.976000    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c\": container with ID starting with 79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c not found: ID does not exist" containerID="79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c"
Sep 11 01:33:25.976069 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.976028    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c"} err="failed to get container status \"79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c\": rpc error: code = NotFound desc = could not find container \"79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c\": container with ID starting with 79fa3e7a573a29fa6dc9375108359fa5a868d8a8e77c7db20ba047da76f0d99c not found: ID does not exist"
Sep 11 01:33:25.998152 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:25.998121    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"]
Sep 11 01:33:26.001859 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:26.001836    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b9930f-546d5b788b-k5kdg"]
Sep 11 01:33:27.110506 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:27.110469    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0" path="/var/lib/kubelet/pods/8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0/volumes"
Sep 11 01:33:32.080167 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:32.080130    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"]
Sep 11 01:33:32.080570 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:32.080350    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy" containerID="cri-o://1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89" gracePeriod=30
Sep 11 01:33:35.295012 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.294974    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-kc22x"]
Sep 11 01:33:35.295499 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.295317    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x" podUID="486bee5f-d3b7-4133-a03a-4f75a4736f7d" containerName="manager" containerID="cri-o://b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6" gracePeriod=30
Sep 11 01:33:35.539478 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.539446    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:33:35.618765 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.618684    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-webhook-cert\") pod \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") "
Sep 11 01:33:35.618930 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.618812    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-metrics-certs\") pod \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") "
Sep 11 01:33:35.618930 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.618834    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l6h7\" (UniqueName: \"kubernetes.io/projected/486bee5f-d3b7-4133-a03a-4f75a4736f7d-kube-api-access-4l6h7\") pod \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\" (UID: \"486bee5f-d3b7-4133-a03a-4f75a4736f7d\") "
Sep 11 01:33:35.620748 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.620716    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "486bee5f-d3b7-4133-a03a-4f75a4736f7d" (UID: "486bee5f-d3b7-4133-a03a-4f75a4736f7d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:33:35.620879 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.620851    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/486bee5f-d3b7-4133-a03a-4f75a4736f7d-kube-api-access-4l6h7" (OuterVolumeSpecName: "kube-api-access-4l6h7") pod "486bee5f-d3b7-4133-a03a-4f75a4736f7d" (UID: "486bee5f-d3b7-4133-a03a-4f75a4736f7d"). InnerVolumeSpecName "kube-api-access-4l6h7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:33:35.620928 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.620872    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "486bee5f-d3b7-4133-a03a-4f75a4736f7d" (UID: "486bee5f-d3b7-4133-a03a-4f75a4736f7d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:33:35.719967 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.719934    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:35.719967 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.719966    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l6h7\" (UniqueName: \"kubernetes.io/projected/486bee5f-d3b7-4133-a03a-4f75a4736f7d-kube-api-access-4l6h7\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:35.720155 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:35.719979    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486bee5f-d3b7-4133-a03a-4f75a4736f7d-webhook-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:36.026108 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:36.026076    2559 generic.go:356] "Generic (PLEG): container finished" podID="486bee5f-d3b7-4133-a03a-4f75a4736f7d" containerID="b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6" exitCode=0
Sep 11 01:33:36.026299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:36.026117    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x" event={"ID":"486bee5f-d3b7-4133-a03a-4f75a4736f7d","Type":"ContainerDied","Data":"b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6"}
Sep 11 01:33:36.026299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:36.026119    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x"
Sep 11 01:33:36.026299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:36.026155    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c6d6b6d8d-kc22x" event={"ID":"486bee5f-d3b7-4133-a03a-4f75a4736f7d","Type":"ContainerDied","Data":"d0a7c0c583fb140ff86154d7ae7c9e185247d2c4dbb86557110da62f1bdc6782"}
Sep 11 01:33:36.026299 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:36.026177    2559 scope.go:117] "RemoveContainer" containerID="b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6"
Sep 11 01:33:36.036003 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:36.035985    2559 scope.go:117] "RemoveContainer" containerID="b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6"
Sep 11 01:33:36.036273 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:33:36.036252    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6\": container with ID starting with b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6 not found: ID does not exist" containerID="b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6"
Sep 11 01:33:36.036313 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:36.036283    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6"} err="failed to get container status \"b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6\": rpc error: code = NotFound desc = could not find container \"b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6\": container with ID starting with b0a100c67bb5c41f7a2e50b3f3c6980f689c77ad4788550d87c76d26125bffd6 not found: ID does not exist"
Sep 11 01:33:36.050931 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:36.050893    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-kc22x"]
Sep 11 01:33:36.060193 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:36.060165    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-kc22x"]
Sep 11 01:33:37.110895 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.110868    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="486bee5f-d3b7-4133-a03a-4f75a4736f7d" path="/var/lib/kubelet/pods/486bee5f-d3b7-4133-a03a-4f75a4736f7d/volumes"
Sep 11 01:33:37.341847 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.341822    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:33:37.436085 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.435998    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-socket\") pod \"294f2c10-5c4f-4c6a-a942-6d092188e168\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") "
Sep 11 01:33:37.436241 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436103    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-token\") pod \"294f2c10-5c4f-4c6a-a942-6d092188e168\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") "
Sep 11 01:33:37.436241 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436138    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-certs\") pod \"294f2c10-5c4f-4c6a-a942-6d092188e168\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") "
Sep 11 01:33:37.436241 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436168    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-podinfo\") pod \"294f2c10-5c4f-4c6a-a942-6d092188e168\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") "
Sep 11 01:33:37.436241 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436199    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-envoy\") pod \"294f2c10-5c4f-4c6a-a942-6d092188e168\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") "
Sep 11 01:33:37.436241 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436226    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vzc4\" (UniqueName: \"kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-kube-api-access-4vzc4\") pod \"294f2c10-5c4f-4c6a-a942-6d092188e168\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") "
Sep 11 01:33:37.436491 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436280    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294f2c10-5c4f-4c6a-a942-6d092188e168-istiod-ca-cert\") pod \"294f2c10-5c4f-4c6a-a942-6d092188e168\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") "
Sep 11 01:33:37.436491 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436303    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-data\") pod \"294f2c10-5c4f-4c6a-a942-6d092188e168\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") "
Sep 11 01:33:37.436491 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436342    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-credential-socket\") pod \"294f2c10-5c4f-4c6a-a942-6d092188e168\" (UID: \"294f2c10-5c4f-4c6a-a942-6d092188e168\") "
Sep 11 01:33:37.436491 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436349    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "294f2c10-5c4f-4c6a-a942-6d092188e168" (UID: "294f2c10-5c4f-4c6a-a942-6d092188e168"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:33:37.436491 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436454    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "294f2c10-5c4f-4c6a-a942-6d092188e168" (UID: "294f2c10-5c4f-4c6a-a942-6d092188e168"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:33:37.436836 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436658    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "294f2c10-5c4f-4c6a-a942-6d092188e168" (UID: "294f2c10-5c4f-4c6a-a942-6d092188e168"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:33:37.436836 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436711    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/294f2c10-5c4f-4c6a-a942-6d092188e168-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "294f2c10-5c4f-4c6a-a942-6d092188e168" (UID: "294f2c10-5c4f-4c6a-a942-6d092188e168"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:33:37.436954 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436840    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294f2c10-5c4f-4c6a-a942-6d092188e168-istiod-ca-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:37.436954 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436852    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-data" (OuterVolumeSpecName: "istio-data") pod "294f2c10-5c4f-4c6a-a942-6d092188e168" (UID: "294f2c10-5c4f-4c6a-a942-6d092188e168"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:33:37.436954 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436859    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-credential-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:37.436954 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436898    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-socket\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:37.436954 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.436915    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-workload-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:37.438435 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.438404    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "294f2c10-5c4f-4c6a-a942-6d092188e168" (UID: "294f2c10-5c4f-4c6a-a942-6d092188e168"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:33:37.438568 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.438441    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "294f2c10-5c4f-4c6a-a942-6d092188e168" (UID: "294f2c10-5c4f-4c6a-a942-6d092188e168"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 01:33:37.438817 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.438795    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-kube-api-access-4vzc4" (OuterVolumeSpecName: "kube-api-access-4vzc4") pod "294f2c10-5c4f-4c6a-a942-6d092188e168" (UID: "294f2c10-5c4f-4c6a-a942-6d092188e168"). InnerVolumeSpecName "kube-api-access-4vzc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:33:37.438900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.438821    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-token" (OuterVolumeSpecName: "istio-token") pod "294f2c10-5c4f-4c6a-a942-6d092188e168" (UID: "294f2c10-5c4f-4c6a-a942-6d092188e168"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:33:37.537513 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.537482    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-podinfo\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:37.537513 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.537509    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-envoy\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:37.537513 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.537518    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vzc4\" (UniqueName: \"kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-kube-api-access-4vzc4\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:37.537747 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.537528    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-data\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:37.537747 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:37.537536    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294f2c10-5c4f-4c6a-a942-6d092188e168-istio-token\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:33:38.041381 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.041342    2559 generic.go:356] "Generic (PLEG): container finished" podID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerID="1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89" exitCode=0
Sep 11 01:33:38.041622 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.041412    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"
Sep 11 01:33:38.041622 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.041427    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" event={"ID":"294f2c10-5c4f-4c6a-a942-6d092188e168","Type":"ContainerDied","Data":"1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89"}
Sep 11 01:33:38.041622 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.041509    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb" event={"ID":"294f2c10-5c4f-4c6a-a942-6d092188e168","Type":"ContainerDied","Data":"5e5eec2f52dc279a8eeb6408b64aeb39036eaaf4ba7d16c8a9aedd1620248ed8"}
Sep 11 01:33:38.041622 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.041533    2559 scope.go:117] "RemoveContainer" containerID="1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89"
Sep 11 01:33:38.054382 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.054356    2559 scope.go:117] "RemoveContainer" containerID="9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7"
Sep 11 01:33:38.067049 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.066505    2559 scope.go:117] "RemoveContainer" containerID="1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89"
Sep 11 01:33:38.067049 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:33:38.066899    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89\": container with ID starting with 1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89 not found: ID does not exist" containerID="1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89"
Sep 11 01:33:38.067049 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.066938    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89"} err="failed to get container status \"1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89\": rpc error: code = NotFound desc = could not find container \"1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89\": container with ID starting with 1445502e34df26b0c43e462e19285d049d74f7f8093a8798acc0b4698f994a89 not found: ID does not exist"
Sep 11 01:33:38.067049 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.066963    2559 scope.go:117] "RemoveContainer" containerID="9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7"
Sep 11 01:33:38.067351 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:33:38.067300    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7\": container with ID starting with 9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7 not found: ID does not exist" containerID="9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7"
Sep 11 01:33:38.067351 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.067335    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7"} err="failed to get container status \"9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7\": rpc error: code = NotFound desc = could not find container \"9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7\": container with ID starting with 9fa0aee5265ebbce5e26711dcd0910d4b203135448acdd6d767cfd87f7898aa7 not found: ID does not exist"
Sep 11 01:33:38.070391 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.070365    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"]
Sep 11 01:33:38.074747 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:38.074723    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-b9930f-openshift-default-5cc6ff4964-q6qbb"]
Sep 11 01:33:39.110521 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:33:39.110488    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" path="/var/lib/kubelet/pods/294f2c10-5c4f-4c6a-a942-6d092188e168/volumes"
Sep 11 01:34:36.969092 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969055    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-x48fd"]
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969443    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b106c2fc-1721-41fe-bdb8-73f150af08a7" containerName="maas-api"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969455    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b106c2fc-1721-41fe-bdb8-73f150af08a7" containerName="maas-api"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969463    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969469    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969478    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969484    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969499    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969504    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969512    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="486bee5f-d3b7-4133-a03a-4f75a4736f7d" containerName="manager"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969519    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="486bee5f-d3b7-4133-a03a-4f75a4736f7d" containerName="manager"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969525    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0" containerName="maas-api"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969530    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0" containerName="maas-api"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969541    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f9e923-15c2-478f-a1da-a089c118cd4e" containerName="maas-api"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969547    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f9e923-15c2-478f-a1da-a089c118cd4e" containerName="maas-api"
Sep 11 01:34:36.969548 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969554    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c61d5e5-6bf2-46bd-b738-5d1d6328ac20" containerName="maas-api"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969559    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c61d5e5-6bf2-46bd-b738-5d1d6328ac20" containerName="maas-api"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969631    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c61d5e5-6bf2-46bd-b738-5d1d6328ac20" containerName="maas-api"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969641    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="486bee5f-d3b7-4133-a03a-4f75a4736f7d" containerName="manager"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969648    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969653    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969659    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969664    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969670    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969675    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ba4c6a2-94a6-465e-acbd-2ba7058bdfb0" containerName="maas-api"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969680    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7f9e923-15c2-478f-a1da-a089c118cd4e" containerName="maas-api"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969685    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b106c2fc-1721-41fe-bdb8-73f150af08a7" containerName="maas-api"
Sep 11 01:34:36.970029 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.969690    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy"
Sep 11 01:34:36.972819 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.972802    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:36.976900 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.976876    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 01:34:36.977060 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.976915    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 01:34:36.977060 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.976924    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 01:34:36.977060 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.976932    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-r4szl\""
Sep 11 01:34:36.983655 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:36.983633    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-x48fd"]
Sep 11 01:34:37.111291 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.111263    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-metrics-certs\") pod \"maas-controller-6c6d6b6d8d-x48fd\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") " pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:37.111468 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.111304    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9t27\" (UniqueName: \"kubernetes.io/projected/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-kube-api-access-j9t27\") pod \"maas-controller-6c6d6b6d8d-x48fd\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") " pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:37.111468 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.111392    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-webhook-cert\") pod \"maas-controller-6c6d6b6d8d-x48fd\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") " pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:37.212490 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.212452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-webhook-cert\") pod \"maas-controller-6c6d6b6d8d-x48fd\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") " pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:37.212835 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.212801    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-metrics-certs\") pod \"maas-controller-6c6d6b6d8d-x48fd\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") " pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:37.212977 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.212895    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9t27\" (UniqueName: \"kubernetes.io/projected/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-kube-api-access-j9t27\") pod \"maas-controller-6c6d6b6d8d-x48fd\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") " pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:37.215081 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.215057    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-webhook-cert\") pod \"maas-controller-6c6d6b6d8d-x48fd\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") " pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:37.215169 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.215060    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-metrics-certs\") pod \"maas-controller-6c6d6b6d8d-x48fd\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") " pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:37.222035 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.221976    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9t27\" (UniqueName: \"kubernetes.io/projected/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-kube-api-access-j9t27\") pod \"maas-controller-6c6d6b6d8d-x48fd\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") " pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:37.284020 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.283982    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:37.557771 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:37.557742    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-x48fd"]
Sep 11 01:34:37.559196 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:34:37.559171    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5053cc5c_b4f4_4290_b8d9_687ce88cc76b.slice/crio-07bafe06d237711ca2210527ec7faff4bf24560397e528f38f5dbb8cb16536c5 WatchSource:0}: Error finding container 07bafe06d237711ca2210527ec7faff4bf24560397e528f38f5dbb8cb16536c5: Status 404 returned error can't find the container with id 07bafe06d237711ca2210527ec7faff4bf24560397e528f38f5dbb8cb16536c5
Sep 11 01:34:38.385335 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:38.385242    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd" event={"ID":"5053cc5c-b4f4-4290-b8d9-687ce88cc76b","Type":"ContainerStarted","Data":"6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83"}
Sep 11 01:34:38.385718 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:38.385343    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd" event={"ID":"5053cc5c-b4f4-4290-b8d9-687ce88cc76b","Type":"ContainerStarted","Data":"07bafe06d237711ca2210527ec7faff4bf24560397e528f38f5dbb8cb16536c5"}
Sep 11 01:34:38.385718 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:38.385370    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:34:38.406964 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:38.406913    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd" podStartSLOduration=1.8935181079999999 podStartE2EDuration="2.40689579s" totalImagesPullingTime="513.377682ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:34:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:34:38.405041638 +0000 UTC m=+2197.919641820" watchObservedRunningTime="2026-09-11 01:34:38.40689579 +0000 UTC m=+2197.921495983"
Sep 11 01:34:49.392669 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:34:49.392638    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:35:00.928073 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:00.928035    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-x48fd"]
Sep 11 01:35:00.928545 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:00.928310    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd" podUID="5053cc5c-b4f4-4290-b8d9-687ce88cc76b" containerName="manager" containerID="cri-o://6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83" gracePeriod=30
Sep 11 01:35:01.169297 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.169274    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:35:01.230844 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.230746    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9t27\" (UniqueName: \"kubernetes.io/projected/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-kube-api-access-j9t27\") pod \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") "
Sep 11 01:35:01.231006 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.230846    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-webhook-cert\") pod \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") "
Sep 11 01:35:01.231006 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.230875    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-metrics-certs\") pod \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\" (UID: \"5053cc5c-b4f4-4290-b8d9-687ce88cc76b\") "
Sep 11 01:35:01.233218 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.233191    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5053cc5c-b4f4-4290-b8d9-687ce88cc76b" (UID: "5053cc5c-b4f4-4290-b8d9-687ce88cc76b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:35:01.233373 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.233325    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-kube-api-access-j9t27" (OuterVolumeSpecName: "kube-api-access-j9t27") pod "5053cc5c-b4f4-4290-b8d9-687ce88cc76b" (UID: "5053cc5c-b4f4-4290-b8d9-687ce88cc76b"). InnerVolumeSpecName "kube-api-access-j9t27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:35:01.233373 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.233356    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5053cc5c-b4f4-4290-b8d9-687ce88cc76b" (UID: "5053cc5c-b4f4-4290-b8d9-687ce88cc76b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:35:01.331998 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.331964    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-webhook-cert\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:35:01.331998 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.331991    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:35:01.331998 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.332000    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9t27\" (UniqueName: \"kubernetes.io/projected/5053cc5c-b4f4-4290-b8d9-687ce88cc76b-kube-api-access-j9t27\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:35:01.517414 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.517380    2559 generic.go:356] "Generic (PLEG): container finished" podID="5053cc5c-b4f4-4290-b8d9-687ce88cc76b" containerID="6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83" exitCode=0
Sep 11 01:35:01.517582 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.517429    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd"
Sep 11 01:35:01.517582 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.517435    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd" event={"ID":"5053cc5c-b4f4-4290-b8d9-687ce88cc76b","Type":"ContainerDied","Data":"6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83"}
Sep 11 01:35:01.517582 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.517487    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c6d6b6d8d-x48fd" event={"ID":"5053cc5c-b4f4-4290-b8d9-687ce88cc76b","Type":"ContainerDied","Data":"07bafe06d237711ca2210527ec7faff4bf24560397e528f38f5dbb8cb16536c5"}
Sep 11 01:35:01.517582 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.517517    2559 scope.go:117] "RemoveContainer" containerID="6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83"
Sep 11 01:35:01.527270 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.527250    2559 scope.go:117] "RemoveContainer" containerID="6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83"
Sep 11 01:35:01.527547 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:35:01.527528    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83\": container with ID starting with 6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83 not found: ID does not exist" containerID="6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83"
Sep 11 01:35:01.527627 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.527557    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83"} err="failed to get container status \"6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83\": rpc error: code = NotFound desc = could not find container \"6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83\": container with ID starting with 6aaa879641d55319cb7b184c539f37e976a93728aae6138a68887611df1d5e83 not found: ID does not exist"
Sep 11 01:35:01.539755 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.539731    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-x48fd"]
Sep 11 01:35:01.543648 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:01.543625    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-x48fd"]
Sep 11 01:35:02.345305 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.345272    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-2jnv7"]
Sep 11 01:35:02.345857 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.345834    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy"
Sep 11 01:35:02.345857 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.345857    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="30003f72-6e7b-4bbc-97fe-9e4686383515" containerName="istio-proxy"
Sep 11 01:35:02.346062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.345886    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5053cc5c-b4f4-4290-b8d9-687ce88cc76b" containerName="manager"
Sep 11 01:35:02.346062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.345896    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5053cc5c-b4f4-4290-b8d9-687ce88cc76b" containerName="manager"
Sep 11 01:35:02.346062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.345913    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy"
Sep 11 01:35:02.346062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.345922    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c73e0d3-eb1d-4a07-b7ad-6a96a13dfcb5" containerName="istio-proxy"
Sep 11 01:35:02.346062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.345936    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy"
Sep 11 01:35:02.346062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.345945    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="294f2c10-5c4f-4c6a-a942-6d092188e168" containerName="istio-proxy"
Sep 11 01:35:02.346062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.346040    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5053cc5c-b4f4-4290-b8d9-687ce88cc76b" containerName="manager"
Sep 11 01:35:02.350592 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.350573    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.354863 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.354840    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 01:35:02.354961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.354840    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 01:35:02.354961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.354844    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-r4szl\""
Sep 11 01:35:02.354961 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.354886    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 01:35:02.359640 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.359593    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-2jnv7"]
Sep 11 01:35:02.444562 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.444516    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0bea30ed-9d9b-4239-b007-34548b306cd8-webhook-cert\") pod \"maas-controller-6c6d6b6d8d-2jnv7\" (UID: \"0bea30ed-9d9b-4239-b007-34548b306cd8\") " pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.444562 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.444568    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmlbj\" (UniqueName: \"kubernetes.io/projected/0bea30ed-9d9b-4239-b007-34548b306cd8-kube-api-access-xmlbj\") pod \"maas-controller-6c6d6b6d8d-2jnv7\" (UID: \"0bea30ed-9d9b-4239-b007-34548b306cd8\") " pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.444803 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.444651    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bea30ed-9d9b-4239-b007-34548b306cd8-metrics-certs\") pod \"maas-controller-6c6d6b6d8d-2jnv7\" (UID: \"0bea30ed-9d9b-4239-b007-34548b306cd8\") " pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.545779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.545745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0bea30ed-9d9b-4239-b007-34548b306cd8-webhook-cert\") pod \"maas-controller-6c6d6b6d8d-2jnv7\" (UID: \"0bea30ed-9d9b-4239-b007-34548b306cd8\") " pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.545967 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.545793    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmlbj\" (UniqueName: \"kubernetes.io/projected/0bea30ed-9d9b-4239-b007-34548b306cd8-kube-api-access-xmlbj\") pod \"maas-controller-6c6d6b6d8d-2jnv7\" (UID: \"0bea30ed-9d9b-4239-b007-34548b306cd8\") " pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.545967 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.545828    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bea30ed-9d9b-4239-b007-34548b306cd8-metrics-certs\") pod \"maas-controller-6c6d6b6d8d-2jnv7\" (UID: \"0bea30ed-9d9b-4239-b007-34548b306cd8\") " pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.548110 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.548087    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0bea30ed-9d9b-4239-b007-34548b306cd8-webhook-cert\") pod \"maas-controller-6c6d6b6d8d-2jnv7\" (UID: \"0bea30ed-9d9b-4239-b007-34548b306cd8\") " pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.548256 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.548231    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0bea30ed-9d9b-4239-b007-34548b306cd8-metrics-certs\") pod \"maas-controller-6c6d6b6d8d-2jnv7\" (UID: \"0bea30ed-9d9b-4239-b007-34548b306cd8\") " pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.555583 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.555555    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmlbj\" (UniqueName: \"kubernetes.io/projected/0bea30ed-9d9b-4239-b007-34548b306cd8-kube-api-access-xmlbj\") pod \"maas-controller-6c6d6b6d8d-2jnv7\" (UID: \"0bea30ed-9d9b-4239-b007-34548b306cd8\") " pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.661674 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.661545    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:02.936877 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.936850    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c6d6b6d8d-2jnv7"]
Sep 11 01:35:02.937862 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:35:02.937831    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bea30ed_9d9b_4239_b007_34548b306cd8.slice/crio-a430b420b64953ed1f1d6b1f2d958bbf42e3012dc7eef0bc6618c9d8dca0b9a5 WatchSource:0}: Error finding container a430b420b64953ed1f1d6b1f2d958bbf42e3012dc7eef0bc6618c9d8dca0b9a5: Status 404 returned error can't find the container with id a430b420b64953ed1f1d6b1f2d958bbf42e3012dc7eef0bc6618c9d8dca0b9a5
Sep 11 01:35:02.939236 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:02.939216    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 01:35:03.111772 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:03.111733    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5053cc5c-b4f4-4290-b8d9-687ce88cc76b" path="/var/lib/kubelet/pods/5053cc5c-b4f4-4290-b8d9-687ce88cc76b/volumes"
Sep 11 01:35:03.531441 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:03.531402    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7" event={"ID":"0bea30ed-9d9b-4239-b007-34548b306cd8","Type":"ContainerStarted","Data":"ec75d019e37c3ae25bdfce44596dffc81e6a6167b5ddd1860fc129ac5cdad13f"}
Sep 11 01:35:03.531850 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:03.531464    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7" event={"ID":"0bea30ed-9d9b-4239-b007-34548b306cd8","Type":"ContainerStarted","Data":"a430b420b64953ed1f1d6b1f2d958bbf42e3012dc7eef0bc6618c9d8dca0b9a5"}
Sep 11 01:35:03.531850 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:03.531620    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:35:03.553026 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:03.552957    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7" podStartSLOduration=1.152182788 podStartE2EDuration="1.552937245s" totalImagesPullingTime="400.754457ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:35:03.54878246 +0000 UTC m=+2223.063382642" watchObservedRunningTime="2026-09-11 01:35:03.552937245 +0000 UTC m=+2223.067537430"
Sep 11 01:35:14.538860 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:35:14.538828    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c6d6b6d8d-2jnv7"
Sep 11 01:38:08.673580 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.673543    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"]
Sep 11 01:38:08.677037 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.677019    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.679496 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.679475    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-d3e6934d-kserve-self-signed-certs\""
Sep 11 01:38:08.684899 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.684877    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"]
Sep 11 01:38:08.812770 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.812730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tls-certs\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.812955 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.812815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-dshm\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.812955 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.812934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-model-cache\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.813076 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.812974    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkj89\" (UniqueName: \"kubernetes.io/projected/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-kube-api-access-nkj89\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.813076 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.812996    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-home\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.813076 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.813039    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tmp-dir\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.914571 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.914536    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-model-cache\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.914571 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.914576    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkj89\" (UniqueName: \"kubernetes.io/projected/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-kube-api-access-nkj89\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.914861 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.914595    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-home\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.914861 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.914734    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tmp-dir\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.914861 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.914780    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tls-certs\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.914987 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.914868    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-dshm\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.915026 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.915015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-home\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.915083 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.915059    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-model-cache\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.915266 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.915241    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tmp-dir\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.917128 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.917108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-dshm\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.917540 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.917524    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tls-certs\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.922037 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.922012    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkj89\" (UniqueName: \"kubernetes.io/projected/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-kube-api-access-nkj89\") pod \"e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") " pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:08.987789 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:08.987704    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:09.259894 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:09.259852    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"]
Sep 11 01:38:09.261235 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:38:09.261201    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e80ea03_1d1e_47e0_98b9_d4a74d9478c4.slice/crio-20b5b213a978cc816d5d7f9618123ca9a89d7c8683696c394e91a3358b3370cf WatchSource:0}: Error finding container 20b5b213a978cc816d5d7f9618123ca9a89d7c8683696c394e91a3358b3370cf: Status 404 returned error can't find the container with id 20b5b213a978cc816d5d7f9618123ca9a89d7c8683696c394e91a3358b3370cf
Sep 11 01:38:09.583386 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:09.583354    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np" event={"ID":"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4","Type":"ContainerStarted","Data":"5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94"}
Sep 11 01:38:09.583386 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:09.583395    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np" event={"ID":"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4","Type":"ContainerStarted","Data":"20b5b213a978cc816d5d7f9618123ca9a89d7c8683696c394e91a3358b3370cf"}
Sep 11 01:38:09.601626 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:09.601558    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np" podStartSLOduration=1.6015434480000001 podStartE2EDuration="1.601543448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:38:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:38:09.599245583 +0000 UTC m=+2409.113845765" watchObservedRunningTime="2026-09-11 01:38:09.601543448 +0000 UTC m=+2409.116143645"
Sep 11 01:38:18.987864 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:18.987827    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:18.988408 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:18.987889    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:19.000995 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:19.000961    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:19.649941 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:19.649909    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:31.889177 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:31.888106    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"]
Sep 11 01:38:46.911712 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:46.911667    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np" podUID="7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" containerName="main" containerID="cri-o://5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94" gracePeriod=45
Sep 11 01:38:47.156675 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.156647    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:47.187024 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.186947    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-model-cache\") pod \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") "
Sep 11 01:38:47.187024 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187001    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tls-certs\") pod \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") "
Sep 11 01:38:47.187210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187024    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tmp-dir\") pod \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") "
Sep 11 01:38:47.187210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187087    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-dshm\") pod \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") "
Sep 11 01:38:47.187210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187156    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkj89\" (UniqueName: \"kubernetes.io/projected/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-kube-api-access-nkj89\") pod \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") "
Sep 11 01:38:47.187360 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187214    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" (UID: "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:38:47.187360 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187229    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-home\") pod \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\" (UID: \"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4\") "
Sep 11 01:38:47.187360 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187342    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" (UID: "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:38:47.187519 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187458    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-home" (OuterVolumeSpecName: "home") pod "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" (UID: "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:38:47.187681 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187656    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-home\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:38:47.187681 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187682    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-model-cache\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:38:47.187877 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.187697    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tmp-dir\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:38:47.189323 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.189294    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-kube-api-access-nkj89" (OuterVolumeSpecName: "kube-api-access-nkj89") pod "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" (UID: "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4"). InnerVolumeSpecName "kube-api-access-nkj89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:38:47.189323 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.189314    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" (UID: "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:38:47.189456 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.189316    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-dshm" (OuterVolumeSpecName: "dshm") pod "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" (UID: "7e80ea03-1d1e-47e0-98b9-d4a74d9478c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 01:38:47.289225 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.289184    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-tls-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:38:47.289225 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.289217    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-dshm\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:38:47.289225 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.289226    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkj89\" (UniqueName: \"kubernetes.io/projected/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4-kube-api-access-nkj89\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:38:47.796708 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.796670    2559 generic.go:356] "Generic (PLEG): container finished" podID="7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" containerID="5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94" exitCode=0
Sep 11 01:38:47.796883 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.796722    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"
Sep 11 01:38:47.796883 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.796744    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np" event={"ID":"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4","Type":"ContainerDied","Data":"5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94"}
Sep 11 01:38:47.796883 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.796795    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np" event={"ID":"7e80ea03-1d1e-47e0-98b9-d4a74d9478c4","Type":"ContainerDied","Data":"20b5b213a978cc816d5d7f9618123ca9a89d7c8683696c394e91a3358b3370cf"}
Sep 11 01:38:47.796883 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.796818    2559 scope.go:117] "RemoveContainer" containerID="5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94"
Sep 11 01:38:47.806543 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.806520    2559 scope.go:117] "RemoveContainer" containerID="5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94"
Sep 11 01:38:47.806845 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:38:47.806825    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94\": container with ID starting with 5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94 not found: ID does not exist" containerID="5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94"
Sep 11 01:38:47.806933 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.806852    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94"} err="failed to get container status \"5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94\": rpc error: code = NotFound desc = could not find container \"5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94\": container with ID starting with 5f89e0ab6ef40511dd68cfbe473ac1143a2b61bf236315dfd68a529c2a6c8a94 not found: ID does not exist"
Sep 11 01:38:47.818153 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.818123    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"]
Sep 11 01:38:47.821925 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:47.821899    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-d3e6934d-kserve-7744559995-7f9np"]
Sep 11 01:38:49.110339 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:49.110310    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" path="/var/lib/kubelet/pods/7e80ea03-1d1e-47e0-98b9-d4a74d9478c4/volumes"
Sep 11 01:38:51.206875 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.206847    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"]
Sep 11 01:38:51.207307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.207254    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" containerName="main"
Sep 11 01:38:51.207307 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.207265    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" containerName="main"
Sep 11 01:38:51.207383 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.207330    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e80ea03-1d1e-47e0-98b9-d4a74d9478c4" containerName="main"
Sep 11 01:38:51.210390 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.210369    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.220946 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.220920    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"]
Sep 11 01:38:51.225366 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.225345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f862f219-169c-4dc1-8023-0b79fa3568e9-openshift-service-ca\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.225478 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.225387    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f862f219-169c-4dc1-8023-0b79fa3568e9-metrics-certs\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.225478 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.225431    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxl9p\" (UniqueName: \"kubernetes.io/projected/f862f219-169c-4dc1-8023-0b79fa3568e9-kube-api-access-sxl9p\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.225551 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.225519    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f862f219-169c-4dc1-8023-0b79fa3568e9-maas-api-tls\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.326216 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.326179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f862f219-169c-4dc1-8023-0b79fa3568e9-maas-api-tls\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.326399 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.326250    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f862f219-169c-4dc1-8023-0b79fa3568e9-openshift-service-ca\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.326399 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.326294    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f862f219-169c-4dc1-8023-0b79fa3568e9-metrics-certs\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.326518 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.326405    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxl9p\" (UniqueName: \"kubernetes.io/projected/f862f219-169c-4dc1-8023-0b79fa3568e9-kube-api-access-sxl9p\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.327062 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.327028    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f862f219-169c-4dc1-8023-0b79fa3568e9-openshift-service-ca\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.328849 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.328828    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f862f219-169c-4dc1-8023-0b79fa3568e9-metrics-certs\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.329026 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.329004    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f862f219-169c-4dc1-8023-0b79fa3568e9-maas-api-tls\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.333934 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.333905    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxl9p\" (UniqueName: \"kubernetes.io/projected/f862f219-169c-4dc1-8023-0b79fa3568e9-kube-api-access-sxl9p\") pod \"maas-api-fcdb7dc4c-vgdqk\" (UID: \"f862f219-169c-4dc1-8023-0b79fa3568e9\") " pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.524343 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.524312    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:51.810553 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.810527    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"]
Sep 11 01:38:51.812102 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:38:51.812073    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf862f219_169c_4dc1_8023_0b79fa3568e9.slice/crio-74a42e88740c875989c6f221b9335f2c6cab8f25b60538b286cff0327d214aa9 WatchSource:0}: Error finding container 74a42e88740c875989c6f221b9335f2c6cab8f25b60538b286cff0327d214aa9: Status 404 returned error can't find the container with id 74a42e88740c875989c6f221b9335f2c6cab8f25b60538b286cff0327d214aa9
Sep 11 01:38:51.822425 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:51.822395    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk" event={"ID":"f862f219-169c-4dc1-8023-0b79fa3568e9","Type":"ContainerStarted","Data":"74a42e88740c875989c6f221b9335f2c6cab8f25b60538b286cff0327d214aa9"}
Sep 11 01:38:52.829304 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:52.829266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk" event={"ID":"f862f219-169c-4dc1-8023-0b79fa3568e9","Type":"ContainerStarted","Data":"adfca1a5c2d4c5689d79903f61b1caa1509c3373e0c46f127b9552b4581b0fa2"}
Sep 11 01:38:52.829768 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:52.829420    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:52.848568 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:52.848521    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk" podStartSLOduration=1.43206167 podStartE2EDuration="1.848508211s" totalImagesPullingTime="416.446541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 01:38:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:38:52.845203167 +0000 UTC m=+2452.359803350" watchObservedRunningTime="2026-09-11 01:38:52.848508211 +0000 UTC m=+2452.363108405"
Sep 11 01:38:58.840087 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:58.840056    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-fcdb7dc4c-vgdqk"
Sep 11 01:38:58.887833 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:58.887802    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"]
Sep 11 01:38:58.888100 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:38:58.888058    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4" podUID="7149ce46-ba29-4b77-8297-6b1325db3127" containerName="maas-api" containerID="cri-o://08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983" gracePeriod=30
Sep 11 01:39:00.147273 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.147244    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:39:00.211571 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.211490    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7149ce46-ba29-4b77-8297-6b1325db3127-openshift-service-ca\") pod \"7149ce46-ba29-4b77-8297-6b1325db3127\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") "
Sep 11 01:39:00.211571 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.211529    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-maas-api-tls\") pod \"7149ce46-ba29-4b77-8297-6b1325db3127\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") "
Sep 11 01:39:00.211916 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.211631    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-metrics-certs\") pod \"7149ce46-ba29-4b77-8297-6b1325db3127\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") "
Sep 11 01:39:00.211916 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.211732    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4j4x\" (UniqueName: \"kubernetes.io/projected/7149ce46-ba29-4b77-8297-6b1325db3127-kube-api-access-n4j4x\") pod \"7149ce46-ba29-4b77-8297-6b1325db3127\" (UID: \"7149ce46-ba29-4b77-8297-6b1325db3127\") "
Sep 11 01:39:00.212012 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.211911    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7149ce46-ba29-4b77-8297-6b1325db3127-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7149ce46-ba29-4b77-8297-6b1325db3127" (UID: "7149ce46-ba29-4b77-8297-6b1325db3127"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 01:39:00.212365 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.212099    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7149ce46-ba29-4b77-8297-6b1325db3127-openshift-service-ca\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:39:00.214025 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.213997    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7149ce46-ba29-4b77-8297-6b1325db3127-kube-api-access-n4j4x" (OuterVolumeSpecName: "kube-api-access-n4j4x") pod "7149ce46-ba29-4b77-8297-6b1325db3127" (UID: "7149ce46-ba29-4b77-8297-6b1325db3127"). InnerVolumeSpecName "kube-api-access-n4j4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 01:39:00.214184 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.214168    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7149ce46-ba29-4b77-8297-6b1325db3127" (UID: "7149ce46-ba29-4b77-8297-6b1325db3127"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:39:00.214229 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.214191    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7149ce46-ba29-4b77-8297-6b1325db3127" (UID: "7149ce46-ba29-4b77-8297-6b1325db3127"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 01:39:00.313207 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.313172    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4j4x\" (UniqueName: \"kubernetes.io/projected/7149ce46-ba29-4b77-8297-6b1325db3127-kube-api-access-n4j4x\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:39:00.313207 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.313201    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-maas-api-tls\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:39:00.313207 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.313210    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7149ce46-ba29-4b77-8297-6b1325db3127-metrics-certs\") on node \"ip-10-0-138-14.ec2.internal\" DevicePath \"\""
Sep 11 01:39:00.877912 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.877875    2559 generic.go:356] "Generic (PLEG): container finished" podID="7149ce46-ba29-4b77-8297-6b1325db3127" containerID="08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983" exitCode=0
Sep 11 01:39:00.878082 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.877921    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"
Sep 11 01:39:00.878082 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.877962    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4" event={"ID":"7149ce46-ba29-4b77-8297-6b1325db3127","Type":"ContainerDied","Data":"08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983"}
Sep 11 01:39:00.878082 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.878012    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4" event={"ID":"7149ce46-ba29-4b77-8297-6b1325db3127","Type":"ContainerDied","Data":"8e1d866a2d1a8d87bdf1634a9d7d0710be2ba4eb93fea915a6575bd16982f55f"}
Sep 11 01:39:00.878082 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.878039    2559 scope.go:117] "RemoveContainer" containerID="08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983"
Sep 11 01:39:00.887754 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.887737    2559 scope.go:117] "RemoveContainer" containerID="08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983"
Sep 11 01:39:00.887994 ip-10-0-138-14 kubenswrapper[2559]: E0911 01:39:00.887975    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983\": container with ID starting with 08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983 not found: ID does not exist" containerID="08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983"
Sep 11 01:39:00.888065 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.888008    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983"} err="failed to get container status \"08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983\": rpc error: code = NotFound desc = could not find container \"08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983\": container with ID starting with 08d4c2132924707587a297cc8fde673c6509a8c89166518762a7effa944f9983 not found: ID does not exist"
Sep 11 01:39:00.899912 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.899889    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"]
Sep 11 01:39:00.903560 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:00.903538    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7b7ff8bc87-5rmg4"]
Sep 11 01:39:01.110570 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:01.110535    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7149ce46-ba29-4b77-8297-6b1325db3127" path="/var/lib/kubelet/pods/7149ce46-ba29-4b77-8297-6b1325db3127/volumes"
Sep 11 01:39:27.110913 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:27.110880    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vxhs5_9d5a1311-f9b5-4cb9-8728-20aca33506d9/manager/0.log"
Sep 11 01:39:27.370144 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:27.370066    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c6d6b6d8d-2jnv7_0bea30ed-9d9b-4239-b007-34548b306cd8/manager/0.log"
Sep 11 01:39:27.977709 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:27.977683    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bd47c7d5-qkzmm_4c001ce3-70a0-427f-8da3-8242f5f7ceb7/copy-manifests/0.log"
Sep 11 01:39:27.984484 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:27.984459    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bd47c7d5-qkzmm_4c001ce3-70a0-427f-8da3-8242f5f7ceb7/manager/0.log"
Sep 11 01:39:28.448441 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:28.448411    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-fcdb7dc4c-vgdqk_f862f219-169c-4dc1-8023-0b79fa3568e9/maas-api/0.log"
Sep 11 01:39:28.799693 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:28.799656    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-pnvcw_9e30efd1-a473-46f9-835f-2e458a737970/postgres/0.log"
Sep 11 01:39:29.742152 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:29.742120    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-xb4vc_6695ef9d-66d4-4bd2-90c5-f62f77948ae8/manager/0.log"
Sep 11 01:39:30.750777 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:30.750741    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn_855298c1-828a-44c3-8656-727732d1722c/istio-proxy/0.log"
Sep 11 01:39:30.875048 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:30.875013    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qrb6l_0b19cad6-8942-43ea-aa5f-01d6bba6f992/discovery/0.log"
Sep 11 01:39:30.989553 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:30.989515    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58fc6d6988-7j7gd_2854327b-d4ef-4d48-96d2-d30fbf98952f/kube-auth-proxy/0.log"
Sep 11 01:39:31.231248 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:31.231187    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f72st_be6b052f-904e-4764-950e-fb94ac2d1571/istio-proxy/1.log"
Sep 11 01:39:31.481395 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:31.481311    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t27rf_8312527e-f390-4329-bd4c-e76f9cf8c530/payload-processing/0.log"
Sep 11 01:39:31.950526 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:31.950501    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qqbzg_c523034c-228e-4c99-a96c-2b63e3ba25c5/main/0.log"
Sep 11 01:39:32.318159 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:32.318125    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hs58k_9377bffc-4a48-4e4e-9576-9c5f461f3805/main/0.log"
Sep 11 01:39:40.901675 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:40.901640    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gg5rc_2484980c-48af-426f-b3c2-95afc18ed2a2/global-pull-secret-syncer/0.log"
Sep 11 01:39:41.099744 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:41.099710    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zndzf_472cc964-fb4d-4f63-bd39-3fdbbcaf868f/konnectivity-agent/0.log"
Sep 11 01:39:41.119378 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:41.119351    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-14.ec2.internal_e42a195d24144df11416cc5ea52d2a36/haproxy/0.log"
Sep 11 01:39:45.778918 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:45.778880    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-xb4vc_6695ef9d-66d4-4bd2-90c5-f62f77948ae8/manager/0.log"
Sep 11 01:39:47.713071 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:47.713040    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1355c87f-5299-4893-aa7d-e7f07535079b/alertmanager/0.log"
Sep 11 01:39:47.746599 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:47.746569    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1355c87f-5299-4893-aa7d-e7f07535079b/config-reloader/0.log"
Sep 11 01:39:47.774834 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:47.774801    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1355c87f-5299-4893-aa7d-e7f07535079b/kube-rbac-proxy-web/0.log"
Sep 11 01:39:47.800527 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:47.800499    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1355c87f-5299-4893-aa7d-e7f07535079b/kube-rbac-proxy/0.log"
Sep 11 01:39:47.828145 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:47.828122    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1355c87f-5299-4893-aa7d-e7f07535079b/kube-rbac-proxy-metric/0.log"
Sep 11 01:39:47.850846 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:47.850823    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1355c87f-5299-4893-aa7d-e7f07535079b/prom-label-proxy/0.log"
Sep 11 01:39:47.870578 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:47.870556    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1355c87f-5299-4893-aa7d-e7f07535079b/init-config-reloader/0.log"
Sep 11 01:39:47.943507 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:47.943479    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-p6l84_5e993eca-e4a0-4c53-a347-4a6258649c74/kube-state-metrics/0.log"
Sep 11 01:39:47.959744 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:47.959716    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-p6l84_5e993eca-e4a0-4c53-a347-4a6258649c74/kube-rbac-proxy-main/0.log"
Sep 11 01:39:47.977562 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:47.977498    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-p6l84_5e993eca-e4a0-4c53-a347-4a6258649c74/kube-rbac-proxy-self/0.log"
Sep 11 01:39:48.035703 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.035676    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-rqrlm_8b51c10d-691c-4769-a8cc-55d8f726b9a0/monitoring-plugin/0.log"
Sep 11 01:39:48.202698 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.202671    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kc2t5_64e628aa-d490-4d30-8d9f-bdf3f6997008/node-exporter/0.log"
Sep 11 01:39:48.235210 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.235133    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kc2t5_64e628aa-d490-4d30-8d9f-bdf3f6997008/kube-rbac-proxy/0.log"
Sep 11 01:39:48.262779 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.262758    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kc2t5_64e628aa-d490-4d30-8d9f-bdf3f6997008/init-textfile/0.log"
Sep 11 01:39:48.632032 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.632002    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-bn4l7_dff58d4d-619a-422a-9f40-76ff0940abde/prometheus-operator/0.log"
Sep 11 01:39:48.650594 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.650573    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-bn4l7_dff58d4d-619a-422a-9f40-76ff0940abde/kube-rbac-proxy/0.log"
Sep 11 01:39:48.676824 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.676799    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-zp4wz_111574e8-558e-4a8d-bd09-cd35c6a2aba1/prometheus-operator-admission-webhook/0.log"
Sep 11 01:39:48.707355 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.707328    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f98bb8f98-qnxp7_539b9ba0-a678-4b77-bd9c-71bd51985803/telemeter-client/0.log"
Sep 11 01:39:48.728434 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.728413    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f98bb8f98-qnxp7_539b9ba0-a678-4b77-bd9c-71bd51985803/reload/0.log"
Sep 11 01:39:48.747310 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.747289    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f98bb8f98-qnxp7_539b9ba0-a678-4b77-bd9c-71bd51985803/kube-rbac-proxy/0.log"
Sep 11 01:39:48.779960 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.779927    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79f995d456-4zrm9_676ff702-be28-4f99-9d85-b000a9af43ed/thanos-query/0.log"
Sep 11 01:39:48.800876 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.800854    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79f995d456-4zrm9_676ff702-be28-4f99-9d85-b000a9af43ed/kube-rbac-proxy-web/0.log"
Sep 11 01:39:48.820598 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.820562    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79f995d456-4zrm9_676ff702-be28-4f99-9d85-b000a9af43ed/kube-rbac-proxy/0.log"
Sep 11 01:39:48.842738 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.842712    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79f995d456-4zrm9_676ff702-be28-4f99-9d85-b000a9af43ed/prom-label-proxy/0.log"
Sep 11 01:39:48.866889 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.866860    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79f995d456-4zrm9_676ff702-be28-4f99-9d85-b000a9af43ed/kube-rbac-proxy-rules/0.log"
Sep 11 01:39:48.889486 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:48.889409    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79f995d456-4zrm9_676ff702-be28-4f99-9d85-b000a9af43ed/kube-rbac-proxy-metrics/0.log"
Sep 11 01:39:51.505444 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.505408    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"]
Sep 11 01:39:51.505838 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.505823    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7149ce46-ba29-4b77-8297-6b1325db3127" containerName="maas-api"
Sep 11 01:39:51.505838 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.505835    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7149ce46-ba29-4b77-8297-6b1325db3127" containerName="maas-api"
Sep 11 01:39:51.505908 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.505904    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7149ce46-ba29-4b77-8297-6b1325db3127" containerName="maas-api"
Sep 11 01:39:51.509103 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.509088    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.511656 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.511632    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sjvqr\"/\"kube-root-ca.crt\""
Sep 11 01:39:51.512560 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.512541    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sjvqr\"/\"default-dockercfg-dv4qx\""
Sep 11 01:39:51.512657 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.512548    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sjvqr\"/\"openshift-service-ca.crt\""
Sep 11 01:39:51.518009 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.517985    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"]
Sep 11 01:39:51.520462 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.520443    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-57655f9cf9-jh6r8_8ed02274-604a-46cb-a096-38c281ec478b/console/0.log"
Sep 11 01:39:51.564085 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.564057    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-p5cwr_b8c6b379-1a03-4d54-ad83-ae2217a4df4f/download-server/0.log"
Sep 11 01:39:51.594046 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.594021    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-proc\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.594172 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.594115    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-sys\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.594172 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.594155    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-lib-modules\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.594264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.594212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jn75\" (UniqueName: \"kubernetes.io/projected/9d088b81-c87e-45da-bd75-9309e8cd6f68-kube-api-access-5jn75\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.594264 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.594251    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-podres\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.695656 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.695593    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-proc\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.695804 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.695719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-sys\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.695804 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.695721    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-proc\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.695804 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.695748    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-lib-modules\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.695804 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.695770    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-sys\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.695957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.695813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jn75\" (UniqueName: \"kubernetes.io/projected/9d088b81-c87e-45da-bd75-9309e8cd6f68-kube-api-access-5jn75\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.695957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.695833    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-podres\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.695957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.695880    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-lib-modules\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.695957 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.695933    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d088b81-c87e-45da-bd75-9309e8cd6f68-podres\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.703656 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.703625    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jn75\" (UniqueName: \"kubernetes.io/projected/9d088b81-c87e-45da-bd75-9309e8cd6f68-kube-api-access-5jn75\") pod \"perf-node-gather-daemonset-gvh7s\" (UID: \"9d088b81-c87e-45da-bd75-9309e8cd6f68\") " pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:51.820450 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:51.820419    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:52.090524 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:52.090498    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"]
Sep 11 01:39:52.092574 ip-10-0-138-14 kubenswrapper[2559]: W0911 01:39:52.092547    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9d088b81_c87e_45da_bd75_9309e8cd6f68.slice/crio-d2df0d3d55f109cc2c9b7e9355fba1a07a77d905bade984a06024e51ac3d08ee WatchSource:0}: Error finding container d2df0d3d55f109cc2c9b7e9355fba1a07a77d905bade984a06024e51ac3d08ee: Status 404 returned error can't find the container with id d2df0d3d55f109cc2c9b7e9355fba1a07a77d905bade984a06024e51ac3d08ee
Sep 11 01:39:52.161753 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:52.161726    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s" event={"ID":"9d088b81-c87e-45da-bd75-9309e8cd6f68","Type":"ContainerStarted","Data":"2626c12bb514415923e54acb18accd5e41d9c973a96f4ab130a0a49e87bb3fa7"}
Sep 11 01:39:52.161879 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:52.161776    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s" event={"ID":"9d088b81-c87e-45da-bd75-9309e8cd6f68","Type":"ContainerStarted","Data":"d2df0d3d55f109cc2c9b7e9355fba1a07a77d905bade984a06024e51ac3d08ee"}
Sep 11 01:39:52.161879 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:52.161855    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:52.177703 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:52.177655    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s" podStartSLOduration=1.177638873 podStartE2EDuration="1.177638873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 01:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 01:39:52.17602281 +0000 UTC m=+2511.690622993" watchObservedRunningTime="2026-09-11 01:39:52.177638873 +0000 UTC m=+2511.692239095"
Sep 11 01:39:53.380087 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:53.380010    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qkfrr_4113c6a8-603c-4e15-a001-047284c67098/dns/0.log"
Sep 11 01:39:53.400585 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:53.400555    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qkfrr_4113c6a8-603c-4e15-a001-047284c67098/kube-rbac-proxy/0.log"
Sep 11 01:39:53.488346 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:53.488318    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-twtj4_a2aee08b-0ce8-4ef5-87f8-c3f06d9f97ec/dns-node-resolver/0.log"
Sep 11 01:39:54.152718 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:54.152688    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hrx2c_0a670085-8845-40b7-ab78-2d028806840c/node-ca/0.log"
Sep 11 01:39:55.038177 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:55.038150    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn_855298c1-828a-44c3-8656-727732d1722c/istio-proxy/0.log"
Sep 11 01:39:55.544259 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:55.544223    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qrb6l_0b19cad6-8942-43ea-aa5f-01d6bba6f992/discovery/0.log"
Sep 11 01:39:55.562745 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:55.562719    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58fc6d6988-7j7gd_2854327b-d4ef-4d48-96d2-d30fbf98952f/kube-auth-proxy/0.log"
Sep 11 01:39:55.616149 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:55.616122    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f72st_be6b052f-904e-4764-950e-fb94ac2d1571/istio-proxy/0.log"
Sep 11 01:39:55.621006 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:55.620984    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f72st_be6b052f-904e-4764-950e-fb94ac2d1571/istio-proxy/1.log"
Sep 11 01:39:55.699545 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:55.699517    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t27rf_8312527e-f390-4329-bd4c-e76f9cf8c530/payload-processing/0.log"
Sep 11 01:39:56.642817 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:56.642792    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tzp2p_6f715154-de02-4259-b992-a7acf4e82e5f/serve-healthcheck-canary/0.log"
Sep 11 01:39:57.095978 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:57.095948    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jfgl6_efecf1f1-ec51-4c26-bd4a-56f81bce1f52/kube-rbac-proxy/0.log"
Sep 11 01:39:57.114914 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:57.114884    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jfgl6_efecf1f1-ec51-4c26-bd4a-56f81bce1f52/exporter/0.log"
Sep 11 01:39:57.134702 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:57.134672    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jfgl6_efecf1f1-ec51-4c26-bd4a-56f81bce1f52/extractor/0.log"
Sep 11 01:39:58.178499 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:58.178469    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sjvqr/perf-node-gather-daemonset-gvh7s"
Sep 11 01:39:59.299492 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:59.299458    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vxhs5_9d5a1311-f9b5-4cb9-8728-20aca33506d9/manager/0.log"
Sep 11 01:39:59.591315 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:59.591235    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c6d6b6d8d-2jnv7_0bea30ed-9d9b-4239-b007-34548b306cd8/manager/0.log"
Sep 11 01:39:59.871454 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:59.871371    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bd47c7d5-qkzmm_4c001ce3-70a0-427f-8da3-8242f5f7ceb7/manager/0.log"
Sep 11 01:39:59.891506 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:39:59.891483    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8bd47c7d5-qkzmm_4c001ce3-70a0-427f-8da3-8242f5f7ceb7/copy-manifests/0.log"
Sep 11 01:40:02.012336 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:02.012250    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn_855298c1-828a-44c3-8656-727732d1722c/istio-proxy/0.log"
Sep 11 01:40:02.506976 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:02.506945    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qrb6l_0b19cad6-8942-43ea-aa5f-01d6bba6f992/discovery/0.log"
Sep 11 01:40:02.568204 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:02.568171    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58fc6d6988-7j7gd_2854327b-d4ef-4d48-96d2-d30fbf98952f/kube-auth-proxy/0.log"
Sep 11 01:40:02.614348 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:02.614319    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f72st_be6b052f-904e-4764-950e-fb94ac2d1571/istio-proxy/0.log"
Sep 11 01:40:02.619202 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:02.619167    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f72st_be6b052f-904e-4764-950e-fb94ac2d1571/istio-proxy/1.log"
Sep 11 01:40:02.699804 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:02.699772    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t27rf_8312527e-f390-4329-bd4c-e76f9cf8c530/payload-processing/0.log"
Sep 11 01:40:04.408389 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:04.408354    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-cbjdn_855298c1-828a-44c3-8656-727732d1722c/istio-proxy/0.log"
Sep 11 01:40:04.916268 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:04.916232    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qrb6l_0b19cad6-8942-43ea-aa5f-01d6bba6f992/discovery/0.log"
Sep 11 01:40:04.933840 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:04.933809    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58fc6d6988-7j7gd_2854327b-d4ef-4d48-96d2-d30fbf98952f/kube-auth-proxy/0.log"
Sep 11 01:40:04.966936 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:04.966909    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f72st_be6b052f-904e-4764-950e-fb94ac2d1571/istio-proxy/0.log"
Sep 11 01:40:04.972155 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:04.972120    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f72st_be6b052f-904e-4764-950e-fb94ac2d1571/istio-proxy/1.log"
Sep 11 01:40:05.038816 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:05.038750    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t27rf_8312527e-f390-4329-bd4c-e76f9cf8c530/payload-processing/0.log"
Sep 11 01:40:07.714537 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:07.714507    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-52jsv_820c3bfc-578f-41c3-97c1-5a5a356345a8/kube-multus/0.log"
Sep 11 01:40:08.155276 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:08.155248    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tgf62_dd3fdc46-edd4-4833-baca-c730c52fed08/kube-multus-additional-cni-plugins/0.log"
Sep 11 01:40:08.191105 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:08.191081    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tgf62_dd3fdc46-edd4-4833-baca-c730c52fed08/egress-router-binary-copy/0.log"
Sep 11 01:40:08.227024 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:08.227000    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tgf62_dd3fdc46-edd4-4833-baca-c730c52fed08/cni-plugins/0.log"
Sep 11 01:40:08.262953 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:08.262931    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tgf62_dd3fdc46-edd4-4833-baca-c730c52fed08/bond-cni-plugin/0.log"
Sep 11 01:40:08.285583 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:08.285559    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tgf62_dd3fdc46-edd4-4833-baca-c730c52fed08/routeoverride-cni/0.log"
Sep 11 01:40:08.304157 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:08.304135    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tgf62_dd3fdc46-edd4-4833-baca-c730c52fed08/whereabouts-cni-bincopy/0.log"
Sep 11 01:40:08.322453 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:08.322426    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tgf62_dd3fdc46-edd4-4833-baca-c730c52fed08/whereabouts-cni/0.log"
Sep 11 01:40:08.453066 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:08.452985    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7qkdk_4d739055-efd4-41fe-9725-197913a5257c/network-metrics-daemon/0.log"
Sep 11 01:40:08.466650 ip-10-0-138-14 kubenswrapper[2559]: I0911 01:40:08.466626    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7qkdk_4d739055-efd4-41fe-9725-197913a5257c/kube-rbac-proxy/0.log"